Log Export
Send platform logs to Splunk, so audit and operational data land in the same place as the rest of your estate.
Streams
Two streams are available and they never overlap, so you can export one without the other.
| Stream | Contents |
|---|---|
audit | Authenticated actions and their outcome: who acted, what they acted on, whether it was allowed or denied, and the tenant it happened in. |
platform | Container logs from platform workloads, plus per-request logs from the service mesh. Operational detail, at far higher volume than audit. Logs from the monitoring, kube-system and argocd namespaces are not collected, so the observability stack's own logs and the cluster control plane are outside this stream. |
Audit events reach Splunk only through the audit stream. Enabling platform does not
duplicate them.
The HEC token
There is no values field for the token: values end up in the platform's own configuration
objects, so a token there would sit in plain text where anyone with cluster read access could
find it. Instead you place the token in the secret store, and the platform fetches it into the
monitoring namespace on your behalf.
Where you put it depends on which store the platform is configured with. In both cases the name and key must match what the platform looks for, or the fetch finds nothing.
Using your own secret store
With secrets.useSecretsProvider: true — OpenBao, Vault or another supported provider — add
the token to that store, at the path the platform reads:
| Key | cm-platform-log-export |
| Property | splunk-hec-token |
Both are configurable under splunkHec.token if the token already lives elsewhere in your
store, which saves moving it.
Using the in-platform store
With secrets.useSecretsProvider: false the store is the cm-platform-secrets namespace in
the cluster itself. Create the Secret there, named for the key above with the property as its
data key:
kubectl -n cm-platform-secrets create secret generic cm-platform-log-export \
--from-literal=splunk-hec-token=<HEC-token>
Checking it arrived
The platform reconciles the token into a splunk-hec-token Secret in monitoring once a
Splunk stream is enabled. If the collector will not start, this is the first thing to look at —
a name that doesn't match leaves it unfulfilled:
kubectl -n monitoring get externalsecret splunk-hec-token
SecretSynced means the token was found.
When you rotate the token, update it in the store and then restart the collector. The platform picks the new value up within a few minutes, but the collector reads it at startup, so a running one keeps sending the old token until it restarts:
kubectl -n monitoring rollout restart deploy otel-collector-gateway-collector
TLS
Splunk Cloud endpoints present a publicly-trusted certificate and need nothing further. A
self-hosted Splunk usually presents one from your own CA, or the self-signed certificate it
ships with; either way, add that CA as a ConfigMap in the monitoring namespace:
kubectl -n monitoring create configmap splunk-ca --from-file=ca.crt=/path/to/ca.crt
If you don't have the certificate to hand, read it off the endpoint:
openssl s_client -connect splunk.example.com:8088 -showcerts </dev/null 2>/dev/null \
| openssl x509 > ca.crt
Configuration
monitoring:
logExport:
splunkHec:
enabled: true
endpoint: https://splunk.example.com:8088/services/collector
# Only needed for a private CA.
tls:
ca:
configMap:
name: splunk-ca
streams:
audit:
enabled: true
index: cm_audit
platform:
enabled: false
An index left empty uses whatever default index the HEC token is configured with. Each
stream also takes a sourcetype, defaulting to cm:audit and cm:logs.
Retaining audit events in the platform
Audit events can additionally be kept in the platform's own store:
monitoring:
logExport:
internal:
audit:
clickhouse: true
This is independent of Splunk — enable both to keep a local copy, or leave it off and treat Splunk as the system of record.
Delivery
Events are buffered in memory and retried when Splunk is unavailable, which covers restarts and brief network faults. Events still buffered when a collector pod is replaced are lost, so if you need a stricter guarantee, keep the local copy as well.
Other destinations
Destinations without a section of their own can be configured directly as OpenTelemetry Collector exporters, per stream:
monitoring:
logExport:
exporters:
audit:
otlphttp/audit:
endpoint: https://collector.example.com:4318
Only the exporters built into the platform's collector are available: otlp and otlphttp
for anything that speaks OpenTelemetry, including a collector of your own that forwards onward,
and debug for checking what a stream contains. Naming anything else stops the whole
collector from starting, so a destination that needs its own protocol needs platform support —
contact us. The same applies to options that need a credential, a private CA, or a delivery
guarantee, since those reach parts of the collector this passthrough cannot.
An exporter ID has to be unique across both streams. The same ID under audit and platform
collides in the generated configuration.