ServiceMetrics Schema

This schema defines the structure of time series metrics returned by the metrics endpoint.

Structure

interface ServiceMetrics {
  timeRange: {
    start: number;
    end: number;
    step: string;
  };
  metrics: {
    totalRequests: TimeSeriesPoint[];
    total4xxErrors: TimeSeriesPoint[];
    total5xxErrors: TimeSeriesPoint[];
    totalTimeouts: TimeSeriesPoint[];
    p50Latency: TimeSeriesPoint[];
    p90Latency: TimeSeriesPoint[];
    p99Latency: TimeSeriesPoint[];
    totalPromptTokens: TimeSeriesPoint[];
    totalCompletionTokens: TimeSeriesPoint[];
    totalTokens: TimeSeriesPoint[];
  };
}

interface TimeSeriesPoint {
  timestamp: number;
  value: number;
}

Field Descriptions

ServiceMetrics

Field	Type	Description
timeRange	object	Information about the time range and resolution of the metrics
timeRange.start	number	Start time of the metrics range (Unix timestamp in seconds)
timeRange.end	number	End time of the metrics range (Unix timestamp in seconds)
timeRange.step	string	Time resolution/step size (e.g., "1m", "5m", "1h")
metrics	object	Container for all metric time series

Metrics Object

Field	Type	Description
totalRequests	TimeSeriesPoint[]	Total number of requests per time bucket
total4xxErrors	TimeSeriesPoint[]	Total number of 4xx client errors per time bucket
total5xxErrors	TimeSeriesPoint[]	Total number of 5xx server errors per time bucket
totalTimeouts	TimeSeriesPoint[]	Total number of timeout errors (504) per time bucket
p50Latency	TimeSeriesPoint[]	50th percentile (median) response latency in milliseconds
p90Latency	TimeSeriesPoint[]	90th percentile response latency in milliseconds
p99Latency	TimeSeriesPoint[]	99th percentile response latency in milliseconds
totalPromptTokens	TimeSeriesPoint[]	Total number of prompt tokens used (for LLM services)
totalCompletionTokens	TimeSeriesPoint[]	Total number of completion tokens generated (for LLM services)
totalTokens	TimeSeriesPoint[]	Total number of tokens (prompt + completion) used (for LLM services)

TimeSeriesPoint

Field	Type	Description
timestamp	number	Unix timestamp in seconds for this data point
value	number	The metric value at this timestamp

Supported Time Steps

The following time step values are supported:

• 1m: 1 minute intervals
• 5m: 5 minute intervals
• 15m: 15 minute intervals
• 30m: 30 minute intervals
• 1h: 1 hour intervals (default)
• 2h: 2 hour intervals
• 6h: 6 hour intervals
• 12h: 12 hour intervals
• 1d: 1 day intervals

Usage

This schema is used by the GET /metrics/{serviceId}/timeseries endpoint which provides time series metrics for service monitoring and observability.

Example Response

{
  "timeRange": {
    "start": 1642248000,
    "end": 1642334400,
    "step": "1h"
  },
  "metrics": {
    "totalRequests": [
      { "timestamp": 1642248000, "value": 150 },
      { "timestamp": 1642251600, "value": 200 },
      { "timestamp": 1642255200, "value": 180 }
    ],
    "total4xxErrors": [
      { "timestamp": 1642248000, "value": 5 },
      { "timestamp": 1642251600, "value": 8 },
      { "timestamp": 1642255200, "value": 3 }
    ],
    "total5xxErrors": [
      { "timestamp": 1642248000, "value": 2 },
      { "timestamp": 1642251600, "value": 1 },
      { "timestamp": 1642255200, "value": 0 }
    ],
    "totalTimeouts": [
      { "timestamp": 1642248000, "value": 1 },
      { "timestamp": 1642251600, "value": 0 },
      { "timestamp": 1642255200, "value": 0 }
    ],
    "p50Latency": [
      { "timestamp": 1642248000, "value": 250 },
      { "timestamp": 1642251600, "value": 280 },
      { "timestamp": 1642255200, "value": 220 }
    ],
    "p90Latency": [
      { "timestamp": 1642248000, "value": 450 },
      { "timestamp": 1642251600, "value": 500 },
      { "timestamp": 1642255200, "value": 400 }
    ],
    "p99Latency": [
      { "timestamp": 1642248000, "value": 800 },
      { "timestamp": 1642251600, "value": 950 },
      { "timestamp": 1642255200, "value": 750 }
    ],
    "totalPromptTokens": [
      { "timestamp": 1642248000, "value": 15000 },
      { "timestamp": 1642251600, "value": 20000 },
      { "timestamp": 1642255200, "value": 18000 }
    ],
    "totalCompletionTokens": [
      { "timestamp": 1642248000, "value": 5000 },
      { "timestamp": 1642251600, "value": 6500 },
      { "timestamp": 1642255200, "value": 6000 }
    ],
    "totalTokens": [
      { "timestamp": 1642248000, "value": 20000 },
      { "timestamp": 1642251600, "value": 26500 },
      { "timestamp": 1642255200, "value": 24000 }
    ]
  }
}

Data Source

Metrics are generated from request logs stored in ClickHouse, specifically from the request_logs table. The following log attributes are used:

• response_code: HTTP response status code
• duration_ms: Request duration in milliseconds
• usage_prompt_tokens: Number of prompt tokens used
• usage_completion_tokens: Number of completion tokens generated
• usage_total_tokens: Total number of tokens used
• path: Request path (for filtering)
• jwt_payload_b64: JWT payload for API key filtering

Query Parameters

The metrics endpoint accepts the following query parameters:

• start: Start time (Unix timestamp in seconds)
• end: End time (Unix timestamp in seconds)
• step: Time resolution (default: "1h")
• apikey: Filter by specific API key
• pathFilter: Filter by request path pattern

Token Metrics

Token-related metrics (totalPromptTokens, totalCompletionTokens, totalTokens) are primarily relevant for LLM (Large Language Model) services and will be zero or low for other service types.

Performance Considerations

• Smaller time steps (1m, 5m) over large time ranges may result in slower queries
• Using path filtering can improve query performance
• API key filtering requires JWT decoding and may impact performance
• Default time range is 24 hours if not specified

Error Handling

If the metrics query fails, the endpoint will return appropriate HTTP status codes:

• 400 Bad Request: Invalid query parameters (e.g., invalid time range or step)
• 403 Forbidden: Insufficient permissions to access service metrics
• 500 Internal Server Error: Server-side error during metrics aggregation