Skip to main content

Tenant Metrics & Usage Tracking

Overview​

The Tenant Metrics system provides comprehensive token usage tracking and visualization. It supports both UI-based monitoring and programmatic API access for billing integration, cost analysis, and usage monitoring.

Key Concepts​

What Gets Tracked​

The platform automatically tracks the following metrics:

  • Prompt Tokens: Tokens sent in requests (input)
  • Completion Tokens: Tokens returned in responses (output)
  • Embedding Tokens: Tokens used for embedding operations
  • Rerank Tokens: Tokens used for reranking operations. They appear in single-key usage views and API key exports.
  • Total Tokens: Sum of prompt and completion tokens

Viewing Levels​

The metrics system provides three levels of granularity:

  1. Platform Level: Aggregate view of all tenants
  2. Tenant Level: Individual tenant's overall usage
  3. Service Level: Per-service breakdown within a tenant

How It Works​

Architecture​

The metrics system is built on ClickHouse for high-performance time-series data storage and querying.

Data Collection: Services log token usage to ClickHouse after each model call, tracking prompt tokens, completion tokens, and embedding tokens

UI Access Points:

Tenant Administrators:

  • Tenants Page → Metrics Tab: Access tenant-specific usage metrics
  • View Toggle: Switch between tenant-level aggregates and service-level breakdowns

Platform Administrators:

  • Platform Page: Access platform-wide metrics with per-tenant breakdown, including full tenant-level and service-level analysis capabilities

API Endpoints:

  • GET /v1/admin/manager/tenants/{tenantId}/metrics - Tenant-level metrics
  • GET /v1/admin/manager/metrics/{serviceId}/timeseries - Service-level metrics

Using the Platform UI​

Accessing Metrics​

Manager tenant button

For Tenant Admins:

  1. Navigate to the Manage Tenant page
  2. Click Metrics

Tenant admin panel button

For Platform Admins:

  1. Navigate to the Tenant admin panel
  2. Click Metrics or select View metrics for any individual tenant

Time Range Selection​

Preset Ranges:

  • Last 2 hours (5-minute intervals)
  • Last 24 hours (1-hour intervals)
  • Last week (1-day intervals)
  • Last month (1-day intervals)

Custom Range:

  • Use date/time pickers for any custom range
  • System automatically selects appropriate intervals

Automatic Interval Selection:

  • ≤ 4 hours: 5-minute buckets
  • ≤ 2 days: 1-hour buckets
  • > 2 days: 1-day buckets

Platform Level View​

Tenant administration platform metrics

What You See:

  1. Usage Summary Cards: Total tenants, tokens by type
  2. Token Usage Chart: Time-series visualization
  3. Tenant Breakdown Table: Sortable, searchable table with tenant details

Available Actions:

  • Search: Filter tenants by name
  • Sort: Order by any token metric
  • Details Button: Navigate to tenant view

Tenant Level View​

Tenant administration tenant usage

What You See:

  1. Tenant Usage Summary: Four cards with token totals
  2. Token Usage Chart: Time-series with prompt, completion, total tokens
  3. Embedding Token Chart: Separate chart for embeddings

View Toggle:

  • Tenant Level: Overall tenant metrics
  • Service Level: Per-service breakdown

Service Level View​

Tenant administration tenant service usage

What You See:

  1. Services List: All services with token usage in selected range
  2. Per Service: Name, ID, token summary cards, expand/collapse
  3. Service Timeline: Expandable detailed chart for each service

Filtering & Sorting:

  • Search: Find by name or ID
  • Sort: By total/prompt/completion/embedding tokens or name

Special Cases:

  • Deleted Services: Show in red with "Deleted Service" label and the original service id
  • No Usage: Services without usage don't appear

Understanding the Charts​

X-axis: Time format based on interval

  • 5-minute/1-hour: Time (HH:MM)
  • 1-day: Date (Mon DD)

Y-axis: Auto-formatted token counts (K = thousands, M = millions)

Chart Colors:

  • Indigo: Prompt tokens
  • Violet: Completion tokens
  • Grape: Total tokens
  • Teal: Embedding tokens
  • Cyan: Rerank tokens (API key usage chart only)

Exporting API Key Usage​

Tenant administrators can download token usage per API key as a CSV file.

Portal v2​

  1. Open the tenant Usage page.
  2. Set the period with From and To.
  3. Click Export API-key CSV.

The file covers the selected period, which is also included in its name, for example api-key-usage-2026-08-01T000000Z-to-2026-08-06T000000Z.csv. Keys without usage in that period are omitted. See API Key Usage Export for the columns, row structure, ordering, and attribution rules.

Legacy portal​

  1. Open the tenant Metrics page.
  2. Select the time range.
  3. Click Export API key usage next to the time-range controls.

The button is available only to tenant administrators. It downloads the same CSV format for the range selected on the page.

API Reference​

Authentication & Authorization​

All endpoints require authentication. Authorization varies by endpoint:

  • Tenant metrics: Tenant admin for specific tenant OR platform-wide tenant admin
  • Service metrics: Service view permissions OR tenant admin

1. Tenant Metrics Endpoint​

GET /v1/admin/manager/tenants/:tenantId/metrics

Retrieve aggregated token usage for all services within a tenant.

Primary Use Cases:

  • Monitor token usage across an entire tenant over time
  • Analyze usage patterns and trends with time-series data
  • Break down consumption by individual service within the tenant

Path Parameters:

  • tenantId (required): Tenant ID

Query Parameters:

  • start (optional): Unix timestamp in seconds for start time
  • end (optional): Unix timestamp in seconds for end time
  • step (optional): Time aggregation step (default: "1h")
    • Valid: "1m", "5m", "15m", "30m", "1h", "2h", "6h", "12h", "1d", "single"

Response:

{
  "timeRange": {
    "start": 1640995200,
    "end": 1641081600,
    "step": "1h"
  },
  "metrics": {
    "totalPromptTokens": [
      { "timestamp": 1640995200, "value": 45630 },
      { "timestamp": 1640998800, "value": 52100 }
    ],
    "totalCompletionTokens": [
      { "timestamp": 1640995200, "value": 28940 },
      { "timestamp": 1640998800, "value": 35200 }
    ],
    "totalEmbeddingTokens": [
      { "timestamp": 1640995200, "value": 7800 },
      { "timestamp": 1640998800, "value": 9400 }
    ],
    "totalTokens": [
      { "timestamp": 1640995200, "value": 82370 },
      { "timestamp": 1640998800, "value": 96700 }
    ]
  },
  "detailed": {
    "byTarget": {
      "service-uuid-1": {
        "targetId": "service-uuid-1",
        "displayName": "Chat Service",
        "metrics": {
          "totalPromptTokens": [
            { "timestamp": 1640995200, "value": 23150 }
          ],
          "totalCompletionTokens": [
            { "timestamp": 1640995200, "value": 15200 }
          ],
          "totalEmbeddingTokens": [
            { "timestamp": 1640995200, "value": 3900 }
          ],
          "totalTokens": [
            { "timestamp": 1640995200, "value": 42250 }
          ]
        }
      }
    }
  }
}

2. Service Metrics Endpoint​

GET /v1/admin/manager/metrics/:serviceId/timeseries

Retrieve time-series metrics for a specific service, including token usage and performance data.

Primary Use Cases:

  • Token billing analysis
  • Tenant-specific billing filtering
  • API key cost tracking
  • Service cost breakdown

Secondary Use Cases:

  • Performance monitoring (requests, errors, latency)

Path Parameters:

  • serviceId (required): Service ID

Query Parameters:

  • start (optional): Unix timestamp in seconds
  • end (optional): Unix timestamp in seconds
  • step (optional): Time aggregation step
    • Valid: "1m", "5m", "15m", "30m", "1h", "2h", "6h", "12h", "1d", "single"
  • tenant_id (optional): Filter by tenant ID
  • apikey (optional): Filter by API key
  • pathFilter (optional): Filter by request path pattern

Response:

{
  "timeRange": {
    "start": 1640995200,
    "end": 1641081600,
    "step": "1h"
  },
  "metrics": {
    "totalRequests": [
      { "timestamp": 1640995200, "value": 150 },
    ],
    "total4xxErrors": [
      { "timestamp": 1640995200, "value": 5 }
    ],
    "total5xxErrors": [
      { "timestamp": 1640995200, "value": 2 }
    ],
    "totalTimeouts": [
      { "timestamp": 1640995200, "value": 0 }
    ],
    "p50Latency": [
      { "timestamp": 1640995200, "value": 245.5 }
    ],
    "p90Latency": [
      { "timestamp": 1640995200, "value": 512.8 }
    ],
    "p99Latency": [
      { "timestamp": 1640995200, "value": 1024.3 }
    ],
    "totalPromptTokens": [
      { "timestamp": 1640995200, "value": 15420 }
    ],
    "totalCompletionTokens": [
      { "timestamp": 1640995200, "value": 8910 }
    ],
    "totalEmbeddingTokens": [
      { "timestamp": 1640995200, "value": 2500 }
    ],
    "totalTokens": [
      { "timestamp": 1640995200, "value": 26830 }
    ]
  }
}

3. API Key Usage Export​

POST /cmind.metering.v1.UsageService/ExportApiKeyTokenUsage

Download one CSV file with token usage for every attributed API key in a tenant. The caller must be a tenant administrator for that tenant.

Request Fields:

  • tenantId (required): Tenant ID
  • start (required): Start of the time range, as a timestamp
  • end (required): End of the time range, as a timestamp. It must be after start.

The time range includes start and excludes end.

Response Fields:

  • filename: Suggested filename, such as api-key-usage-2026-08-01T000000Z-to-2026-08-06T000000Z.csv
  • csv: CSV file contents as bytes

CSV Columns:

ColumnDescription
period_start_utcStart of the requested range, in UTC
period_end_utcEnd of the requested range, in UTC
api_key_idAPI key ID
row_typetotal for the key's total, or model for one model used by the key
modelModel identifier for a model row; empty for a total row
input_tokensTokens sent in requests
output_tokensTokens returned in responses
embedding_tokensTokens used for embedding
rerank_tokensTokens used for reranking
total_tokensSum of the four token columns above

Each key has one total row followed by one model row for every model it used. The total row is the sum of the key's model rows. Keys are ordered by total tokens, highest first, and models within each key use the same ordering. Ties are ordered by key or model identifier.

The export is not paginated and has no row limit. It includes every API key with attributed usage in the requested range. If no key has usage, the file contains only the header row.

Only usage that the platform can attribute to an API key is included. Requests made without an API key are omitted, so the CSV totals can be lower than the tenant totals shown in the usage views.

The request fails with invalid_argument if start or end is missing, if end is not after start, or if tenantId is not a valid tenant ID.

Usage Examples​

Parameters:

  • {tenant_id}: ID of the tenant
  • {service_id}: ID of the service
  • {api_key}: JWT payload part only
    • Example: For JWT eyJhbG...aaa.eyJleHA...bbb.k7GFW...ccc, use only middle eyJleHA...bbb
  • start/end: Unix timestamps in seconds
  • step: Time bucket size (1m, 5m, 15m, 30m, 1h, 2h, 6h, 12h, 1d, single)

Total tenant usage (simplest):

GET /v1/admin/manager/tenants/{tenant_id}/metrics?start=1704067200&end=1706745600&step=single

Returns summed token totals for entire period.

Detailed tenant billing report:

GET /v1/admin/manager/tenants/{tenant_id}/metrics?start=1704067200&end=1706745600&step=1d

Returns daily token breakdown for entire period.

Tenant usage of specific service:

GET /v1/admin/manager/metrics/{service_id}/timeseries?tenant_id={tenant_id}&start=1704067200&end=1706745600&step=single

API key usage within a tenant:

GET /v1/admin/manager/metrics/{service_id}/timeseries?tenant_id={tenant_id}&apikey={api_key}&start=1704067200&end=1706745600&step=single

Returns total token usage for a specific API key within a tenant.

Step Selection Guide​

Choose appropriate step values based on use case:

  • Simple totals: "single" - One aggregated value
  • Monthly billing: "1d" - Daily totals
  • Weekly analysis: "2h" or "6h" - Spending patterns
  • Real-time monitoring: "1h" - Hourly tracking
  • Detailed audits: "15m" or "30m" - Granular analysis

Error Responses​

Standard HTTP error codes:

  • 400 Bad Request: Missing required parameters
  • 401 Unauthorized: Unauthorized or missing authentication
  • 403 Forbidden: Insufficient permissions
  • 500 Internal Server Error: Server processing error

Data Types​

TimeSeriesPoint​

interface TimeSeriesPoint {
  timestamp: number;
  value: number;
}

TokenMetrics​

interface TokenMetrics {
  totalPromptTokens: TimeSeriesPoint[];
  totalCompletionTokens: TimeSeriesPoint[];
  totalEmbeddingTokens: TimeSeriesPoint[];
  totalTokens: TimeSeriesPoint[];
}

PerformanceMetrics​

interface PerformanceMetrics {
  totalRequests: TimeSeriesPoint[];
  total4xxErrors: TimeSeriesPoint[];
  total5xxErrors: TimeSeriesPoint[];
  totalTimeouts: TimeSeriesPoint[];
  p50Latency: TimeSeriesPoint[];
  p90Latency: TimeSeriesPoint[];
  p99Latency: TimeSeriesPoint[];
}

ServiceMetrics​

interface ServiceMetrics {
  timeRange: {
    start: number;
    end: number;
    step: string;
  };
  metrics: PerformanceMetrics & TokenMetrics;
}

TenantMetrics​

interface TenantMetrics {
  timeRange: {
    start: number;
    end: number;
    step: string;
  };
  metrics: TokenMetrics;
  detailed?: TenantDetailedMetrics;
}

TenantDetailedMetrics​

export interface TenantDetailedMetrics {
  byTarget: {
    [targetId: string]: TenantTargetMetrics;
  };
}

TenantTargetMetrics​

interface TenantTargetMetrics {
  targetId: string;
  displayName?: string;
  metrics: TokenMetrics;
}