Skip to main content

OpenShift/OKD Prerequisites

Additional prerequisites for deploying CM Platform on OpenShift Container Platform (OCP) or OKD.

Air-gapped installations: all manifests referenced below are included in the installation tarball under workstation/openshift-prerequisites/ — apply them from there instead of the URLs.

Istio (OpenShift Service Mesh 3)​

CM Platform uses Istio for service mesh. On OCP, install it via the Red Hat OpenShift Service Mesh 3 operator before running the Helm installation.

1. Install OSSM3 Operator​

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/ossm3-operator.yaml

Wait until the Istio CRDs are available before proceeding. OLM installs the operator asynchronously (this can take a few minutes), and kubectl wait fails with NotFound on a CRD that has not been registered yet — poll for existence first:

until kubectl get crd istios.sailoperator.io >/dev/null 2>&1; do sleep 10; done
kubectl wait --for=condition=Established crd/istios.sailoperator.io --timeout=120s

2. Install Istio CNI​

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/istio-cni.yaml

3. Install Istio​

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/istio.yaml

Node Feature Discovery (NFD)​

NFD is required for GPU node labeling.

1. Install NFD Operator​

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/nfd-operator.yaml

Wait until the NFD CRDs are available before applying the CR (same async-operator pattern as the Istio section — poll for existence, then wait):

until kubectl get crd nodefeaturediscoveries.nfd.openshift.io >/dev/null 2>&1; do sleep 10; done
kubectl wait --for=condition=Established crd/nodefeaturediscoveries.nfd.openshift.io --timeout=120s

2. Apply NFD CR​

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/nfd-cr.yaml

User workload monitoring​

PurplePill GPU metrics are collected by OpenShift user workload monitoring. A cluster administrator must enable it in the existing cluster-monitoring-config ConfigMap before installing CM Platform:

apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-monitoring-config
  namespace: openshift-monitoring
data:
  config.yaml: |
    enableUserWorkload: true

Merge this setting with any existing monitoring configuration; do not replace the ConfigMap's other settings. After installation creates the monitoring namespace, ensure it is not excluded from user workload monitoring:

kubectl label namespace monitoring openshift.io/user-monitoring-

If CM Platform is already installed, apply these settings now; they can be changed at any time. Until user workload monitoring is on and the monitoring namespace is included in it, no GPU metrics are collected.

If the namespace uses default-deny ingress policies, allow OpenShift user workload Prometheus to reach PurplePill pods on TCP port 9531.

MachineConfig Resources​

Use your normal process to apply the below MachineConfig resources to configure worker nodes at the OS level. Apply each manifest before running the CM Platform installation:

(!) The Machine Config Operator will roll out the configuration and reboot affected nodes. Wait until all nodes return to Ready state before proceeding.

For Hosted Control Planes (HyperShift) "how to" see the note below.

99-worker-web-service-mnt-data​

Creates and SELinux-labels /mnt/data on worker nodes for GPU model cache storage.

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/99-worker-web-service-mnt-data.yaml

99-worker-ome-llm-cache​

Creates and SELinux-labels /mnt/llm_cache on worker nodes for OME LLM model cache.

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/99-worker-ome-llm-cache.yaml

99-worker-otel-checkpoint-dir​

Creates and SELinux-labels /var/otel/file-checkpoints on worker nodes for persistent OpenTelemetry filelog receiver checkpoints. Apply this MachineConfig before installing or upgrading CM Platform: the log collector deliberately requires the prepared directory instead of allowing Kubernetes to create it with an incompatible host SELinux label. The manifest targets the worker pool only; if log-collecting nodes belong to another MachineConfigPool, apply a copy with the matching machineconfiguration.openshift.io/role label, or the collector pods on those nodes never start.

kubectl apply -f https://raw.githubusercontent.com/ConfidentialMind/examples/main/platform-deployment/openshift-prerequisites/99-worker-otel-checkpoint-dir.yaml

Updating an existing cluster to this MachineConfig repairs the label on the existing directory and its checkpoint files without deleting them. The service reapplies the label on every subsequent node boot. After the MachineConfigPool finishes updating, verify every node eligible to run the log collector reports container_file_t:s0:

oc debug node/<node> -- chroot /host ls -RZ /var/otel/file-checkpoints

Hosted Control Planes (HyperShift)​

If your OpenShift cluster uses hosted control planes (HyperShift), the standard MachineConfigs are not applicable at the hosted clusters level. Look for Red Hat documentation regarding "Handling machine configuration for hosted control planes", e.g. OCP 4.19: https://docs.redhat.com/en/documentation/openshift_container_platform/4.19/html/hosted_control_planes/handling-machine-configuration-for-hosted-control-planes .

Re-encrypt Routes (optional)​

If your environment requires reencrypt TLS termination at the load balancer level, use the openshift-tls-reencrypt chart. Routes with reencrypt termination require destinationCACertificate, which must be read at runtime from the cm-selfsigned-ca Secret created by cert-manager. Because ArgoCD renders Helm charts via helm template (no cluster access), this chart must be installed directly with helm install/upgrade, where lookup can fetch the current CA.

Run the below after the Platform deployment. Take the "self signed TLS" path when installing the Platform.
Once platform-installer has run and the self-signed CA Secret bacame ready:

helm install \
  openshift-tls-reencrypt oci://confidentialmindpub.azurecr.io/openshift-tls-reencrypt \
  --namespace cm-platform --create-namespace \
  --values basic-values.yaml

Re-run helm upgrade after cert rotation — lookup fetches the current CA on every render. The cm-selfsigned-ca certificate is valid for 10 years and cert-manager renews it a year before it expires, so in normal operation that re-run is needed once a decade.