[bmscomp]

Why

This is Phase 3 (chart side) of the "Kates on any cluster" plan, decision D1: chart defaults should pass a Kyverno-governed cluster's usual policies, with no separate profile to pick.

Measured on main, every chart failed somewhere. Most failures were helm test pods with no resources or seccomp profile; the rest were the kafka-ui Deployment, PostgreSQL's seccomp profile, the kates init container, the chaos experiments Job, and the monitoring subcharts' resources. The repository's Kyverno CI job couldn't catch any of this: it applied the kates policies to a manifest in another namespace, so nothing ever matched.

What changes

• Restricted by default. Every pod each chart renders, including helm test pods, hooks and jobs, now gets:

  • a RuntimeDefault seccomp profile and a non-root user its image runs as;
  • allowPrivilegeEscalation: false and drop: [ALL];
  • requests and a memory limit.

  These are all values, under each chart's existing names.

• Strimzi-managed pods:

  • The Strimzi resource charts (kafka-cluster, connect-cluster, mirror-maker2) write the same settings into the pod templates the operator fills, so their pods pass even on a shared operator.
  • strimzi-operator turns on Strimzi's restricted pod security provider. This is a minor bump, because every operand rolls once.

• podLabels reaches every pod each chart renders; the chart's own labels win a clash. Where a subchart has its own key, the chart documents it, as on the monitoring and Litmus subcharts.

• Registry and pull secrets. global.imageRegistry and the pull-secret values reach every image and pod. Pull secrets are accepted as plain names or as {name: …}.

• Policy exceptions. kyvernoPolicy.exceptions on kates and kates-chaos renders a PolicyException in the release namespace; on kates-chaos it is narrowed to the Litmus pods. The charts' own ClusterPolicies render nothing under rbac.scope=namespace.

• Chaos stress. CPU_STRESS and IO_STRESS now:

  • run a pinned, configurable image: kates.chaos.stress-image, set by the chart's chaos.stressImage, default docker.io/polinux/stress:1.0.4;
  • run under a restricted security context, as the target pod's own user and group. That user comes from the pod, or else from a container, which is where OpenShift sets it, with 65534 as the fallback.

• CI. scripts/check-chart-policies.sh replaces the Kyverno job, which never matched anything. It renders every chart with defaults and with its optional workloads on, and checks every render against four sample policies in .github/kyverno/policies/, including a team label on every pod. It fails on any violation, and on a render the policies don't match at all. The only exceptions are named files: the host-level node-exporter, the Velero node agent, and the Litmus operator and exporter, whose subchart has no security-context value. CI's Kyverno CLI moves from v1.13.0 to v1.16.4 (third commit), the version of the Kyverno that kates deploy installs. v1.13.0 ignores the Pod Security controls those exceptions lift, so the job failed on exactly those three workloads.

• Can't pass, and documented with their switches:

  • node-exporter;
  • Velero's node agent and its label-namespace Job;
  • the Litmus operator and exporter;
  • SeaweedFS with the values the chart ships, and its bucket hook.

  The book also gives the SeaweedFS subchart values that do pass, for a new install.

• Versions. Each chart is bumped past v1.28.0 with a changes entry: kafka-cluster 1.0.12, connect-cluster 2.1.10, mirror-maker2 0.11.8, kafka-common 0.1.1, kates 0.13.0, kates-chaos 2.3.0, strimzi-operator 0.4.0, monitoring 1.7.0, kates-platform 0.8.3, apicurio-registry 0.4.1, kafka-ui 0.3.1, headlamp 0.2.1, legacy-kafka 0.2.1 and velero 11.3.4. kates-platform's subchart floors move to these versions, and its entry says ACTION REQUIRED: make platform-chart-deps.

• kates-chaos GameDay and Helm tests (second commit). They ran sh in registry.k8s.io/kubectl, which has no shell, so they ended in StartError and helm test and gameday.enabled always failed. This bug was already on main. They now run images.tester, ghcr.io/bmscomp/kates-tester:1.28.0, as its uid and gid 1000. scripts/check-versions.sh holds that pin to the kates appVersion, so the release pin PR moves it with the others.

• Book: 17-security (what each chart sets, what can't pass, the check), 20-installation-guide, appendix C (the CI job), and appendices A, B and D. Two MCP caveat refs into charts/kates/values.yaml are re-pointed, plus one into monitoring's Chart.yaml.

Behaviour changes for existing users

• Root images. Test pods, installer and GameDay Jobs that pinned an image needing root stop starting, because they now set runAsNonRoot (kates-chaos 2.3.0, among others).
• global.imageRegistry on kates. It is now put in front of every image the kates chart names, so an image value that already names the mirror gets it twice (kates 0.13.0).
• Memory limits. New limits apply to monitoring's Prometheus and others (monitoring 1.7.0); a busy existing install may need them raised.
• Kaniko builds. A Kaniko Connect build needs an operator without the restricted provider (strimzi-operator 0.4.0).

How it was built and checked

• Two multi-agent workflows. The first built the change, with seven owners on separate files, then CI-check rounds, the book, and four reviewers with three skeptics per finding; 25 findings were confirmed. The second fixed those, re-pointed refs and re-reviewed; 11 more were confirmed. I fixed the last of them myself.
• What the reviews caught, among others:
  • the new check would have been red on every run;
  • kates rejected plain-name pull secrets under the umbrella chart;
  • dropping the kates-chaos mutate policy would have stopped the Litmus operator starting on restricted clusters;
  • a privileged Velero node agent would have rendered an invalid spec;
  • the stress user was wrong on OpenShift;
  • the chart versions collided with v1.28.0.
• Scope pulled back. One agent moved SeaweedFS from hostPath to PVCs, which breaks every existing SeaweedFS release. SeaweedFS is off by default, so that change was reverted, and the book documents the subchart values for a new install instead.
• Local mirror of the CI jobs these paths trigger:
  • scripts/check-chart-policies.sh: 13 charts pass, 0 fail.
  • helm lint on every chart, and kates-platform's render with the new floors.
  • gen-chart-table.sh --check, gen-version-matrix.sh --check, check-versions.sh.
  • shellcheck with CI's flags, and actionlint on ci.yml.
  • CLI go test ./cmd (caveat refs), and golangci-lint --new-from-rev with 0 issues.
  • KubernetesChaosProviderTest with Spotless.
  • The book style check, and a scratch Quarto render with all 8098 fragment links landing.
• Live on kind-panda, in a scratch namespace labelled pod-security.kubernetes.io/enforce=restricted:
  • The kates chart (rbac.scope=namespace) was admitted, and PostgreSQL ran Ready.
  • The Kates API started and restarted because it can't reach Kafka from a scratch namespace. main's chart restarts the same way in an unlabelled namespace, with the same messaging error in the logs.
  • kafka-ui was admitted and waits only for its Strimzi user secret.
  • kates-chaos's rendered GameDay Job and both test pods started their shell scripts as uid 1000. Their checks fail only because the live check ran them without their RBAC (Forbidden for the default service account).

Not done

• Not run locally: helm unittest, because the plugin isn't installed here; CI runs it for kafka-cluster.
• Not exercised live: the Strimzi operator's restricted provider on a running cluster, and the chaos stress container against OpenShift.
• The CLI half of Phase 3 (a kates preflight that reads installed Kyverno policies, and restricted CLI diagnostic pods) and feeding .labels into podLabels come next.