cmind/modelmanager/v1/model_deployment_observability_service.proto
Package: cmind.modelmanager.v1
Enums
MetricsStep
| Value | Number | Description |
|---|
METRICS_STEP_UNSPECIFIED | 0 | |
METRICS_STEP_1M | 1 | |
METRICS_STEP_5M | 2 | |
METRICS_STEP_15M | 3 | |
METRICS_STEP_30M | 4 | |
METRICS_STEP_1H | 5 | |
METRICS_STEP_2H | 6 | |
METRICS_STEP_6H | 7 | |
METRICS_STEP_12H | 8 | |
METRICS_STEP_1D | 9 | |
Messages
GetModelDeploymentGpuOverviewRequest
GetModelDeploymentGpuOverviewResponse
| Field | Type | Number | Description |
|---|
nodes | repeated GpuNode | 1 | |
deployments | repeated ModelDeploymentGpuSummary | 2 | |
GpuNode
| Field | Type | Number | Description |
|---|
nodeName | string | 1 | |
gpus | repeated GpuDevice | 2 | |
GpuDevice
| Field | Type | Number | Description |
|---|
id | string | 1 | |
uuid | string | 2 | |
model | string | 3 | |
totalVramMib | int64 | 4 | |
usedVramMib | optional int64 | 5 | |
freeVramMib | optional int64 | 6 | |
ModelDeploymentGpuSummary
| Field | Type | Number | Description |
|---|
uid | string | 1 | |
name | string | 2 | |
gpus | repeated GpuUsage | 3 | |
GpuUsage
| Field | Type | Number | Description |
|---|
nodeName | string | 1 | |
gpuId | string | 2 | |
gpuUuid | string | 3 | |
usedVramMib | int64 | 4 | |
GetModelDeploymentMetricsRequest
| Field | Type | Number | Description |
|---|
deploymentName | string | 1 | |
startTime | uint32 | 2 | |
endTime | uint32 | 3 | |
step | MetricsStep | 4 | |
GetModelDeploymentMetricsResponse
| Field | Type | Number | Description |
|---|
timeRange | MetricsTimeRange | 1 | |
metrics | ModelDeploymentMetrics | 2 | |
MetricsTimeRange
| Field | Type | Number | Description |
|---|
start | uint32 | 1 | |
end | uint32 | 2 | |
step | MetricsStep | 3 | |
MetricPoint
| Field | Type | Number | Description |
|---|
timestamp | uint32 | 1 | |
value | double | 2 | |
ModelDeploymentMetrics
| Field | Type | Number | Description |
|---|
totalRequests | repeated MetricPoint | 1 | |
total4xxErrors | repeated MetricPoint | 2 | |
total5xxErrors | repeated MetricPoint | 3 | |
p50Latency | repeated MetricPoint | 4 | |
p90Latency | repeated MetricPoint | 5 | |
p99Latency | repeated MetricPoint | 6 | |
totalPromptTokens | repeated MetricPoint | 7 | |
totalCompletionTokens | repeated MetricPoint | 8 | |
totalEmbeddingTokens | repeated MetricPoint | 9 | |
totalTokens | repeated MetricPoint | 10 | |
totalRerankTokens | repeated MetricPoint | 11 | |
ListNodeModelCacheUsageRequest
ListNodeModelCacheUsageResponse
| Field | Type | Number | Description |
|---|
nodes | repeated NodeModelCacheUsage | 1 | Per GPU node holding the model-weight cache. |
NodeModelCacheUsage
NodeModelCacheUsage describes the filesystem backing the model-weight cache on a single node, plus the models cached there.
| Field | Type | Number | Description |
|---|
nodeName | string | 1 | |
cachePath | string | 2 | Cache root on the node (the configured base path, e.g. /mnt/llm_cache). |
capacityBytes | optional int64 | 3 | Filesystem totals for the mount backing the cache root. Optional rather than erroring the whole list: they come from a best-effort metrics source (node-exporter via Prometheus) and are absent when that's unavailable for a node/mount, while the cache listing from Kubernetes stays valid. |
usedBytes | optional int64 | 4 | |
availableBytes | optional int64 | 5 | |
models | repeated CachedModelUsage | 6 | Models cached on this node. |
CachedModelUsage
CachedModelUsage is one model cached on a node and its footprint there. (CachedModel is already taken in this package for the deployment-side model reference, so this carries the Usage suffix.)
| Field | Type | Number | Description |
|---|
modelUrl | string | 1 | |
displayName | string | 2 | |
sizeBytes | optional int64 | 3 | Weight-file size of the model (best-effort, from HuggingFace repo metadata, which tracks the on-disk size closely). Absent when the size can't be determined — a non-HuggingFace source, or a lookup that failed. |
deploymentNames | repeated string | 4 | Deployments referencing this cached model (sorted), so the UI can show when several deployments share one cache. |
Services
ModelDeploymentObservabilityService
GetModelDeploymentGpuOverview
GET /cmind.modelmanager.v1.ModelDeploymentObservabilityService/GetModelDeploymentGpuOverview
Request: GetModelDeploymentGpuOverviewRequest
Response: GetModelDeploymentGpuOverviewResponse
GetModelDeploymentMetrics
GET /cmind.modelmanager.v1.ModelDeploymentObservabilityService/GetModelDeploymentMetrics
Request: GetModelDeploymentMetricsRequest
Response: GetModelDeploymentMetricsResponse
ListNodeModelCacheUsage
GET /cmind.modelmanager.v1.ModelDeploymentObservabilityService/ListNodeModelCacheUsage
ListNodeModelCacheUsage reports the node-local model-weight cache: per-node filesystem capacity/usage and, per node, the cached models and their sizes.
Request: ListNodeModelCacheUsageRequest
Response: ListNodeModelCacheUsageResponse