Size CPU and memory
Each platform service reserves CPU and memory on the node it runs on, and has a memory limit. The
defaults suit a typical installation. Change them with the services setting when your nodes or your
workload need other values.
How the services use the nodes
- Requests are what a pod reserves. Kubernetes only starts a pod on a node that still has that much CPU and memory unreserved.
- Memory limits cap what a pod can use. A pod that goes over its limit is restarted, and
kubectl get podsshowsOOMKilledas the reason. Whatever the pod was doing at that moment fails. - CPU has no limit. A service can use any CPU its node isn't using. When the node is busy, CPU is shared in proportion to each service's request, so each gets roughly what it requested.
A pod that doesn't fit on any node stays Pending. During an upgrade the old pod keeps running, so the
service stays on its previous version until there is room. On a new installation the service doesn't
start.
Defaults
| Service | Key under services | Replicas | CPU request | Memory request | Memory limit |
|---|---|---|---|---|---|
| API keys and request authentication | authn | 2 | 100m | 100Mi | 256Mi |
| Manager | managerGo | 2 | 100m | 300Mi | 768Mi |
| Model Gateway | modelGateway | 2 | 100m | 200Mi | 2Gi |
| File parser for chat attachments | fileparser | 2 | 100m | 700Mi | 2Gi |
| Keycloak to OpenFGA bridge | keycloakFgaBridge | 1 | 10m | 32Mi | 256Mi |
| Admin Portal | portal | 2 | 10m | 64Mi | 256Mi |
| Chat | chat | 2 | 10m | 64Mi | 256Mi |
| Documentation site | docs | 1 | 10m | 32Mi | 256Mi |
| Each RAG endpoint | rag | Set per RAG | 20m | 512Mi | 3Gi |
| Each MCP deployment | mcpDeployment | 1 each | 10m | 64Mi | 512Mi |
Together the services request about 0.9 CPU cores and 2.9Gi of memory, plus 20m and 512Mi for each RAG endpoint, and 10m and 64Mi for each MCP deployment. Other components, such as the databases, Keycloak and ClickHouse, reserve their own CPU and memory on top of this.
Before you install or upgrade, check that the nodes have that much memory unreserved. kubectl describe nodes lists what is already reserved under Allocated resources. Free memory, as kubectl top nodes
reports it, is a different number and doesn't decide whether a pod fits.
Changing the defaults
Set replicas and resources per service under services in your installer values:
services:
modelGateway:
replicas: 3
resources:
limits:
memory: 4Gi
rag:
resources:
requests:
memory: 1Gi
resources takes the standard Kubernetes container shape. Values you leave out keep their defaults,
and null removes one, so limits: {memory: null} runs a service without a memory limit.
rag applies to every RAG endpoint, and changing it restarts all of them. mcpDeployment applies in the
same way to every MCP deployment. rag, mcpDeployment and keycloakFgaBridge take no replicas.
Each RAG sets its own replica count, each MCP deployment runs one replica, and the bridge always runs one
replica.
RAG endpoints restart a few at a time, so that an upgrade or a change to rag.resources doesn't start
all of them at once and exhaust node memory. rag.maxConcurrentRollouts sets how many restart at the same
time. The default is 5, and 0 removes the limit. The remaining RAG endpoints keep serving on their current
version until their turn comes.
Set rag.maxConcurrentRollouts to a whole number. With null or a value that is not a number, the component
that manages models, RAG endpoints and MCP deployments doesn't start, so none of them is created or changed
until you correct the value.
Keep each request at or below its limit. Kubernetes rejects a pod whose request is above its limit.
For rag and mcpDeployment, the component that manages models, RAG endpoints and MCP deployments doesn't
start at all, so none of them is created or changed until you correct the values. If you raise the rag memory request above 3Gi, raise its
limit too.
When to raise a memory limit
If a service keeps restarting with OOMKilled, raise its memory limit. These are the services most
likely to need it:
- RAG endpoints hold each file in memory while indexing it. A very large file can take a RAG over
its limit, and that upload fails. Raise
rag.resources.limits.memory. - File parser holds each document attached in chat in memory while converting it, so several large
documents at once add up. Raise
fileparser.resources.limits.memory. - Model Gateway holds each transcription request in memory, up to 128 MiB of audio. Raise
modelGateway.resources.limits.memoryif many large audio files are transcribed at the same time.
ClickHouse, which stores traces, logs and usage data, has its own setting. See ClickHouse resources.