Skip to content

feat(cli): a git-ignored .labels file labels the namespaces Kates creates - #445

Merged
bmscomp merged 2 commits into
mainfrom
feat/labels-file
Oct 21, 2026
Merged

bmscomp merged 2 commits into
mainfrom
feat/labels-file

Conversation

@bmscomp

@bmscomp bmscomp commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Why

Phase 1 of the "Kates on any cluster" plan. Many shared clusters admit a namespace only when it carries the labels their policies require, such as an owning team or a segment. Today labels come only from --namespace-label or KATES_NAMESPACE_LABELS. Neither is checked, and several namespaces Kates creates never get them: cert-manager, kyverno, the detect probes, kates auto and the migrate labs. On top of that, an existing namespace is relabelled on every run, and an account that may not patch it stops the deploy.

What changes

  • .labels file: one key=value per line, # comments, read from the current directory, or from --labels-file. .gitignore lists it, and .labels.example documents it. Merge order per key: .labels < KATES_NAMESPACE_LABELS < --namespace-label.
  • Validation for every source:
    • Kubernetes label syntax applies; empty values are now allowed.
    • Keys under kubernetes.io, k8s.io, helm.sh, strimzi.io and kates.io are refused, except pod-security.kubernetes.io.
    • Each error names its source, such as .labels:3.
    • kates deploy checks the labels before its pre-flight, and --dry-run lists each label with its source.
  • Namespaces (ensureDeployNamespace):
    • A missing namespace is created in one call, with the labels, managed-by and a kates.io/namespace-labels annotation listing the keys Kates set.
    • An existing namespace is patched only when a label differs. When nothing differs there is no call at all, so an account that can't patch still passes. Managed-by is never added to an existing namespace. A key the annotation lists is removed once it leaves the labels.
    • When RBAC refuses the create or the patch, the error keeps kubectl's message and adds the manifest, or the kubectl label command, for a cluster admin. An admission denial (Gatekeeper, VAP) keeps only its own message, because an admin would be refused the same way.
    • --verbose no longer drops kubectl's stderr from the error.
  • Where else the labels apply:
    • cert-manager and kyverno, when deploy installs them;
    • kates auto, kates kyverno apply, kates detect;
    • lab namespaces from migrate up, migrate run and migrate source deploy, where the lab's own labels win;
    • every kates-detect-* probe namespace, created in one call from a manifest and without Pod Security labels, because the probes' pods aren't restricted-ready. The new --no-write-probes skips those probes;
    • ensure_namespace in scripts/common.sh, which applies ./.labels and does nothing when the file is absent, so CI is unaffected.
  • Book: a "Namespace Labels" section in the Deployment Guide, and the new flags in the CLI reference.

Evidence

  • Local mirror of the triggered CI jobs: go vet, go test ./..., and -race -p 1 on cmd and pkg/detect all pass. golangci-lint --new-from-rev=origin/main reports 0 issues. shellcheck passes with CI's flags. check-book-style.sh passes. A scratch Quarto render passes the fragment check, with 8008 links landing.
  • Review: a four-reviewer pass, with three skeptics per finding, confirmed 14 findings, which come down to 8 distinct problems. All are fixed here:
    • the forbidden handling that swallowed admission denials;
    • the stderr lost under --verbose;
    • Pod Security labels on probe namespaces;
    • kates detect and the bench probe namespaces not wired;
    • untested error branches;
    • an unreadable .labels ignored by the script;
    • cd without a subshell in deploy-kafka-generic.sh;
    • two book overstatements.
  • Live on kind-panda:
    • kates detect with a .labels file created kates-detect-latency-<id> carrying team=payments, and the Secret audit passed through the manifest path.
    • kates deploy --dry-run with all three sources printed cost-center=9999 KATES_NAMESPACE_LABELS, owner=sre --namespace-label and team=payments .labels:1.
    • A .labels with app.kubernetes.io/part-of stopped the deploy with .labels:2: … prefix "app.kubernetes.io" is reserved before any cluster call.

Behaviour changes for existing users

  • KATES_NAMESPACE_LABELS and --namespace-label now accept empty values and refuse reserved prefixes, such as app.kubernetes.io/part-of.
  • app.kubernetes.io/managed-by=kates is no longer stamped on namespaces Kates did not create.

Not done

  • A full kates deploy against a cluster whose policy requires labels was not run; the namespace paths are covered by unit tests and the live detect run.
  • scripts/deploy-kafka-generic.sh still creates the Kafka namespace with kubectl create namespace … | kubectl apply and does not label it, and deploy-monitoring.sh relies on helm --create-namespace. The book's table names only what ensure_namespace does.
  • Pod labels (a podLabels contract carrying .labels to pods) are Phase 3.

…ates

Shared clusters often admit a namespace only with the labels their policies
require. A .labels file (one key=value per line, git-ignored, with a
commented .labels.example) now feeds the same label set as
KATES_NAMESPACE_LABELS and --namespace-label, merged per key in that order.
Every source is held to the Kubernetes label syntax, and keys under
kubernetes.io, k8s.io, helm.sh, strimzi.io and kates.io are refused, except
pod-security.kubernetes.io. kates deploy checks the labels before its
pre-flight, and --dry-run lists each label with its source.

A namespace Kates creates gets the labels, managed-by and a
kates.io/namespace-labels annotation in one create. An existing namespace is
patched only when a label differs, managed-by is never added to it, and a key
the annotation lists is removed once it leaves the labels. When RBAC refuses
the create or the patch, the error keeps kubectl's message and adds the
manifest or the kubectl label command for a cluster admin. An admission
denial keeps only its own message. --verbose no longer hides that message.

The labels also reach cert-manager and kyverno when deploy installs them,
kates auto, kates kyverno apply, kates detect, migrate lab namespaces, the
detect probe namespaces (without Pod Security labels; --no-write-probes skips
those probes) and scripts/common.sh ensure_namespace. --labels-file reads
another file on each of these commands.
… arithmetic

CodeQL's go/allocation-size-overflow flagged make([]string, 0, len(args)+1)
once .labels file data reached kubectl's arguments. The +1 cannot overflow,
but appending to args[:end:end] does the same copy without it. The test now
gives the input spare capacity, so a write into the caller's array shows.
@bmscomp
bmscomp merged commit daad596 into main Oct 21, 2026
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant