Skip to content

CI's Talos validation renders a config production never producesΒ #3854

Description

@devantler

πŸ€– Generated by the Agentic Engineer

Evidence

ci.yaml's validate-talos job renders the machine config with talosctl gen config +
talosctl machineconfig patch, then runs talosctl validate. Production's config is not
produced that way β€” it comes from KSail, which does extra work on machine.install before the
config is ever written.

KSail v7.184.0, the version this repo pins in ci.yaml and cd.yaml, folds
machine.install.extraKernelArgs into the Image Factory schematic and then, in
reconcileFoldedKernelArgs:

  • clears machine.install.extraKernelArgs from the rendered config, and
  • pins machine.install.grubUseUKICmdline = true.

CI's render does neither. So for machine.install, CI validates a config production never
produces.

Impact

This is currently masked because CI pins Talos 1.13.9, where nothing complains. Under the Talos
v1.14 migration (#3850) it stops being cosmetic: the raw render leaves a live v1alpha1
machine.install, which v1.14 rejects as
UnattendedInstallConfig config is incompatible with v1alpha1 config (.machine.install) β€”
so talos/cluster/enable-apparmor.yaml reads as a hard schema blocker in CI when production would
not hit it at that point at all. That misreading is what sent #3851 looking for a Talos-side answer
to a field KSail already handles.

The general form is worse than the specific one: any future divergence between what KSail renders
and what CI renders is invisible, in the job whose whole purpose is to catch bad machine config.

Expected behaviour

The Talos validation gate exercises the configuration production actually deploys, so a patch that
would break production fails CI and a patch that would not, does not.

Acceptance criteria

  1. validate-talos validates a config produced the same way production's is β€” either by invoking
    KSail's generation path, or by applying the same fold (drop extraKernelArgs, pin
    grubUseUKICmdline: true) before validating.
  2. A negative control proves the gate still fails on a genuinely invalid patch β€” a change that only
    ever passes is not a gate.
  3. A positive control proves the new path reproduces the fold: a patch carrying
    extraKernelArgs + grubUseUKICmdline: false validates, and the rendered result shows the field
    cleared and UKI cmdline true.
  4. The KSail version the gate relies on is pinned and stated, so a KSail bump cannot silently change
    what CI validates.

Size and risk

Small-to-medium. Contained to one CI job, but it changes what that job attests, so the controls in
(2) and (3) are the substance rather than ceremony.

Discovered while deciding #3851. Blocks nothing directly, but #3850 is much harder to reason about
until CI and production agree.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions