cmind/modelmanager/v1/model_deployment_observability_service.proto
Package: cmind.modelmanager.v1
Enums
GpuMetricsSource
GpuMetricsSource tells a client whether GPU usage in the overview is live.
| Value | Number | Description |
|---|
GPU_METRICS_SOURCE_UNSPECIFIED | 0 | |
GPU_METRICS_SOURCE_LIVE | 1 | Nodes, capacity and per-deployment usage come from Purplepill through Prometheus. used_vram_mib / free_vram_mib are set where a GPU reports. |
GPU_METRICS_SOURCE_NODE_LABELS | 2 | The platform runs no monitoring stack. Nodes and capacity come from node labels; there is no usage to show and none is expected. |
GPU_METRICS_SOURCE_LIVE_UNAVAILABLE | 3 | Monitoring is enabled but live metrics could not be read (Prometheus unreachable, or GPU nodes with no Purplepill series). Nodes and capacity come from node labels, usage is unset and deployments carry no GPU usage. Clients should say so and offer a retry; the condition is expected to clear. |
MetricsStep
| Value | Number | Description |
|---|
METRICS_STEP_UNSPECIFIED | 0 | |
METRICS_STEP_1M | 1 | |
METRICS_STEP_5M | 2 | |
METRICS_STEP_15M | 3 | |
METRICS_STEP_30M | 4 | |
METRICS_STEP_1H | 5 | |
METRICS_STEP_2H | 6 | |
METRICS_STEP_6H | 7 | |
METRICS_STEP_12H | 8 | |
METRICS_STEP_1D | 9 | |
Messages
GetModelDeploymentGpuOverviewRequest
GetModelDeploymentGpuOverviewResponse
| Field | Type | Number | Description |
|---|
nodes | repeated GpuNode | 1 | |
deployments | repeated ModelDeploymentGpuSummary | 2 | |
metricsSource | GpuMetricsSource | 3 | Where the figures in this response come from. Node inventory is always served; only usage depends on live metrics, and clients must not read absent usage as zero. |
GpuNode
| Field | Type | Number | Description |
|---|
nodeName | string | 1 | |
gpus | repeated GpuDevice | 2 | |
GpuDevice
| Field | Type | Number | Description |
|---|
id | string | 1 | |
uuid | string | 2 | |
model | string | 3 | |
totalVramMib | int64 | 4 | |
usedVramMib | optional int64 | 5 | |
freeVramMib | optional int64 | 6 | |
ModelDeploymentGpuSummary
| Field | Type | Number | Description |
|---|
uid | string | 1 | |
name | string | 2 | |
gpus | repeated GpuUsage | 3 | |
GpuUsage
| Field | Type | Number | Description |
|---|
nodeName | string | 1 | |
gpuId | string | 2 | |
gpuUuid | string | 3 | |
usedVramMib | int64 | 4 | |
GetModelDeploymentMetricsRequest
| Field | Type | Number | Description |
|---|
deploymentName | string | 1 | |
startTime | uint32 | 2 | |
endTime | uint32 | 3 | |
step | MetricsStep | 4 | |
GetModelDeploymentMetricsResponse
| Field | Type | Number | Description |
|---|
timeRange | MetricsTimeRange | 1 | |
metrics | ModelDeploymentMetrics | 2 | |
MetricsTimeRange
| Field | Type | Number | Description |
|---|
start | uint32 | 1 | |
end | uint32 | 2 | |
step | MetricsStep | 3 | |
MetricPoint
| Field | Type | Number | Description |
|---|
timestamp | uint32 | 1 | |
value | double | 2 | |
ModelDeploymentMetrics
| Field | Type | Number | Description |
|---|
totalRequests | repeated MetricPoint | 1 | |
total4xxErrors | repeated MetricPoint | 2 | |
total5xxErrors | repeated MetricPoint | 3 | |
p50Latency | repeated MetricPoint | 4 | |
p90Latency | repeated MetricPoint | 5 | |
p99Latency | repeated MetricPoint | 6 | |
totalPromptTokens | repeated MetricPoint | 7 | |
totalCompletionTokens | repeated MetricPoint | 8 | |
totalEmbeddingTokens | repeated MetricPoint | 9 | |
totalTokens | repeated MetricPoint | 10 | |
totalRerankTokens | repeated MetricPoint | 11 | |
ListNodeModelCacheUsageRequest
ListNodeModelCacheUsageResponse
| Field | Type | Number | Description |
|---|
nodes | repeated NodeModelCacheUsage | 1 | Per GPU node holding the model-weight cache. |
NodeModelCacheUsage
NodeModelCacheUsage describes the filesystem backing the model-weight cache on a single node, plus the models cached there.
| Field | Type | Number | Description |
|---|
nodeName | string | 1 | |
cachePath | string | 2 | Cache root on the node (the configured base path, e.g. /mnt/llm_cache). |
capacityBytes | optional int64 | 3 | Filesystem totals for the mount backing the cache root. Optional rather than erroring the whole list: they come from a best-effort metrics source (node-exporter via Prometheus) and are absent when that's unavailable for a node/mount, while the cache listing from Kubernetes stays valid. |
usedBytes | optional int64 | 4 | |
availableBytes | optional int64 | 5 | |
models | repeated CachedModelUsage | 6 | Models cached on this node. |
CachedModelUsage
CachedModelUsage is one model cached on a node and its footprint there. (CachedModel is already taken in this package for the deployment-side model reference, so this carries the Usage suffix.)
| Field | Type | Number | Description |
|---|
modelUrl | string | 1 | |
displayName | string | 2 | |
sizeBytes | optional int64 | 3 | Weight-file size of the model (best-effort, from HuggingFace repo metadata, which tracks the on-disk size closely). Absent when the size can't be determined — a non-HuggingFace source, or a lookup that failed. |
deploymentNames | repeated string | 4 | Deployments referencing this cached model (sorted), so the UI can show when several deployments share one cache. |
Services
ModelDeploymentObservabilityService
GetModelDeploymentGpuOverview
GET /cmind.modelmanager.v1.ModelDeploymentObservabilityService/GetModelDeploymentGpuOverview
Request: GetModelDeploymentGpuOverviewRequest
Response: GetModelDeploymentGpuOverviewResponse
GetModelDeploymentMetrics
GET /cmind.modelmanager.v1.ModelDeploymentObservabilityService/GetModelDeploymentMetrics
Request: GetModelDeploymentMetricsRequest
Response: GetModelDeploymentMetricsResponse
ListNodeModelCacheUsage
GET /cmind.modelmanager.v1.ModelDeploymentObservabilityService/ListNodeModelCacheUsage
ListNodeModelCacheUsage reports the node-local model-weight cache: per-node filesystem capacity/usage and, per node, the cached models and their sizes.
Request: ListNodeModelCacheUsageRequest
Response: ListNodeModelCacheUsageResponse