CPU and memory

Each platform service reserves CPU and memory on the node it runs on, and has a memory limit. The defaults suit a typical installation. Change them with the services setting when your nodes or your workload need other values.

How the services use the nodes

• Requests are what a pod reserves. Kubernetes only starts a pod on a node that still has that much CPU and memory unreserved.
• Memory limits cap what a pod can use. A pod that goes over its limit is restarted, and kubectl get pods shows OOMKilled as the reason. Whatever the pod was doing at that moment fails.
• CPU has no limit. A service can use any CPU its node isn't using. When the node is busy, CPU is shared in proportion to each service's request, so each gets roughly what it requested.

A pod that doesn't fit on any node stays Pending. During an upgrade the old pod keeps running, so the service stays on its previous version until there is room. On a new installation the service doesn't start.

Defaults

Service	Key under services	Replicas	CPU request	Memory request	Memory limit
API keys and request authentication	authn	2	100m	100Mi	256Mi
Manager	managerGo	2	100m	300Mi	768Mi
Model Gateway	modelGateway	2	100m	200Mi	2Gi
File parser for chat attachments	fileparser	2	100m	700Mi	2Gi
Keycloak to OpenFGA bridge	keycloakFgaBridge	1	10m	32Mi	256Mi
Admin Portal	portal	2	10m	64Mi	256Mi
Chat	chat	2	10m	64Mi	256Mi
Documentation site	docs	1	10m	32Mi	256Mi
Each RAG endpoint	rag	Set per RAG	20m	512Mi	3Gi

Together the services request about 0.9 CPU cores and 2.9Gi of memory, plus 20m and 512Mi for each RAG endpoint. Other components, such as the databases, Keycloak and ClickHouse, reserve their own CPU and memory on top of this.

Before you install or upgrade, check that the nodes have that much memory unreserved. kubectl describe nodes lists what is already reserved under Allocated resources. Free memory, as kubectl top nodes reports it, is a different number and doesn't decide whether a pod fits.

Changing the defaults

Set replicas and resources per service under services in your installer values:

services:
  modelGateway:
    replicas: 3
    resources:
      limits:
        memory: 4Gi
  rag:
    resources:
      requests:
        memory: 1Gi

resources takes the standard Kubernetes container shape. Values you leave out keep their defaults, and null removes one, so limits: {memory: null} runs a service without a memory limit.

rag applies to every RAG endpoint, and changing it restarts all of them. rag and keycloakFgaBridge take resources only. Each RAG sets its own replica count, and the bridge always runs one replica.

Keep each request at or below its limit. Kubernetes rejects a pod whose request is above its limit. For rag, the component that manages models and RAG endpoints doesn't start at all, so no model or RAG endpoint is created or changed until you correct the values. If you raise the rag memory request above 3Gi, raise its limit too.

When to raise a memory limit

If a service keeps restarting with OOMKilled, raise its memory limit. These are the services most likely to need it:

• RAG endpoints hold each file in memory while indexing it. A very large file can take a RAG over its limit, and that upload fails. Raise rag.resources.limits.memory.
• File parser holds each document attached in chat in memory while converting it, so several large documents at once add up. Raise fileparser.resources.limits.memory.
• Model Gateway holds each transcription request in memory, up to 128 MiB of audio. Raise modelGateway.resources.limits.memory if many large audio files are transcribed at the same time.

ClickHouse, which stores traces, logs and usage data, has its own setting. See ClickHouse resources.