GPU Info Schema

This document defines the schema for the GET /gpu-info endpoint, which returns detailed information about GPU resources and usage across the cluster.

GPUInfoResponse

Response for GET /gpu-info.

Properties

Property	Type	Description
nodes	NodeInfo[]	List of nodes with GPU resources
pods	PodInfo[]	List of pods using GPU resources
prometheusMetrics	boolean	Whether Prometheus metrics are available

NodeInfo Object

Property	Type	Description
nodeName	string	Name of the Kubernetes node
hostname	string	Hostname of the node
totalVramMB	number	Total VRAM available on the node in MB
usedVramMB	number	Total VRAM used on the node in MB
gpus	GPU[]	List of GPUs on the node

GPU Object

Property	Type	Description
id	string	GPU ID (index)
uuid	string	Unique UUID of the GPU
vramTotalMB	number	Total VRAM of the GPU in MB
vramUsedMB	number	Used VRAM of the GPU in MB
gpuType	string	Type/Model of the GPU (e.g., "NVIDIA A100")

PodInfo Object

Property	Type	Description
id	string	Pod ID (UID)
podName	string	Name of the pod
nodeName	string	Name of the node the pod is running on
gpuUsage	GPUUsage[]	List of GPU usage details for the pod

GPUUsage Object

Property	Type	Description
id	string	UUID of the GPU being used
vramUsedMB	number	Amount of VRAM used by the pod on this GPU in MB

Example

{
  "nodes": [
    {
      "nodeName": "gke-cluster-pool-1-12345",
      "hostname": "node-1",
      "totalVramMB": 81920,
      "usedVramMB": 40960,
      "gpus": [
        {
          "id": "0",
          "uuid": "GPU-1234-5678-90ab-cdef",
          "vramTotalMB": 40960,
          "vramUsedMB": 20480,
          "gpuType": "NVIDIA A100"
        },
        {
          "id": "1",
          "uuid": "GPU-aaaa-bbbb-cccc-dddd",
          "vramTotalMB": 40960,
          "vramUsedMB": 20480,
          "gpuType": "NVIDIA A100"
        }
      ]
    }
  ],
  "pods": [
    {
      "id": "pod-xyz-789",
      "podName": "model-service-deployment-abc",
      "nodeName": "gke-cluster-pool-1-12345",
      "gpuUsage": [
        {
          "id": "GPU-1234-5678-90ab-cdef",
          "vramUsedMB": 10240
        }
      ]
    }
  ],
  "prometheusMetrics": true
}