Skip to content

Add external minion onboarding script and runbook for salt-minion-vcf - #2

Merged
prawintiru merged 6 commits into
mainfrom
prawin/minion-bootstap
Sep 3, 2026
Merged

Add external minion onboarding script and runbook for salt-minion-vcf#2
prawintiru merged 6 commits into
mainfrom
prawin/minion-bootstap

Conversation

@prawintiru

@prawintiru prawintiru commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator
  • scripts/onboarding/vcf-ops-onboard.py: interactive CLI that resolves a VCF Operations-managed Salt master, starts a salt-minion-vcf instance (Docker or Kubernetes/Helm), and trusts its key against the master via the GET /api/salt/master and POST /api/salt/minions/{id}/trusted-keys VCF Operations APIs - with audit logging, spinners, and a multi-minion loop, while staying fully non-interactive when driven by CLI flags.
  • docs/runbook.md: operational runbook covering end-to-end minion onboarding and how to supply saltext.vcf pillar data (local and master-dispatched paths) for each supported VCF component.

- scripts/onboarding/vcf-ops-onboard.py: interactive CLI that resolves a
  VCF Operations-managed Salt master, starts a salt-minion-vcf instance
  (Docker or Kubernetes/Helm), and trusts its key against the master via
  the GET /api/salt/master and POST /api/salt/minions/{id}/trusted-keys
  VCF Operations APIs - with audit logging, spinners, and a multi-minion
  loop, while staying fully non-interactive when driven by CLI flags.
- docs/runbook.md: operational runbook covering end-to-end minion
  onboarding and how to supply saltext.vcf pillar data (local and
  master-dispatched paths) for each supported VCF component.
- Ignore __pycache__/*.pyc alongside the existing ignore rules.
…trust

Root-caused via live testing against a VCF-managed Salt master:

- FIPS-validated masters don't implement SHA-1 for RSA OAEP/PKCS1v15 at
  all - a minion defaulting to SHA-1 doesn't get a clean rejection, it
  crashes the master's payload handler on every auth attempt. FIPS mode
  (fips_mode: True, OAEP-SHA224/PKCS1v15-SHA224) is now on by default in
  docker-entrypoint.sh, matching real VCF-managed minion config.
- master_finger-based identity verification was unreliable in this
  environment even once FIPS was fixed. VCF's own internal component
  minions never do fingerprint verification at all - they're handed the
  master's public key directly and trust it. The onboarding script now
  pre-seeds SALT_MASTER_PUBKEY_B64 (written to minion_master.pub) for
  Docker minions instead, matching that pattern. master_finger is kept
  for the Kubernetes/Helm path, which doesn't yet support direct pubkey
  seeding.
- Logged full response bodies for get_master_details/add_trusted_key
  (no secrets in either) to make this class of issue diagnosable from
  the audit log directly next time.

Updated docs/external-minion-configuration.md and
scripts/onboarding/README.md to reflect the real fixes in place of
earlier troubleshooting guesses, and added the esxi-cluster-patching,
usb-controller-removal, and vc-patch runbooks.
…trust

Root-caused via live testing against a VCF-managed Salt master:

- FIPS-validated masters don't implement SHA-1 for RSA OAEP/PKCS1v15 at
  all - a minion defaulting to SHA-1 doesn't get a clean rejection, it
  crashes the master's payload handler on every auth attempt. FIPS mode
  (fips_mode: True, OAEP-SHA224/PKCS1v15-SHA224) is now on by default in
  docker-entrypoint.sh, matching real VCF-managed minion config.
- master_finger-based identity verification was unreliable in this
  environment even once FIPS was fixed. VCF's own internal component
  minions never do fingerprint verification at all - they're handed the
  master's public key directly and trust it. The onboarding script now
  pre-seeds SALT_MASTER_PUBKEY_B64 (written to minion_master.pub) for
  Docker minions instead, matching that pattern. master_finger is kept
  for the Kubernetes/Helm path, which doesn't yet support direct pubkey
  seeding.
- Logged full response bodies for get_master_details/add_trusted_key
  (no secrets in either) to make this class of issue diagnosable from
  the audit log directly next time.

Also updates docs/external-minion-configuration.md and
scripts/onboarding/README.md to reflect these fixes in place of the
earlier troubleshooting guesses (this is the commit the previous one
on this branch should have included, but a bad git-add pathspec
silently dropped these files from it).
…ourceKind

vcf-ops-onboard.py now supports --action {configure,rotate,list}:
rotate identifies the target minion by its current public key (read off
the running container/pod's PKI dir) and calls the new
POST /api/salt/minions/rotate API, then pushes the new keypair into the
running instance and restarts it (container recreate for Docker, Secret
update + Pod recreate for Kubernetes); list prints trusted minions via
GET /api/salt/minions.

Also have docker-entrypoint.sh set the vcfops_resource_kind grain to
"external" on every start, so VCF Operations' minion listing reports a
real resourceKind for these externally managed minions instead of null.
FQDN and clear PKI files before every container start

Two related bugs found while validating rotate end-to-end against a real
master:

- Master FQDN resolution: internal-only master hostnames often only
  resolve via a static /etc/hosts entry on the host, which Docker
  containers don't inherit. docker_start() now resolves the master's FQDN
  on the host itself and passes --add-host automatically, so no manual IP
  lookup is needed.

- Stale PKI files: -v {volume}:/etc/salt/pki/minion can be a named volume
  or a bind-mounted host path, either of which outlives `docker rm`.
  docker-entrypoint.sh only seeds the keypair when minion.pem is absent
  (so a genuine restart keeps its identity), so reusing a volume/path
  across separate configure/rotate runs silently kept an OLD keypair
  instead of the one just registered via the API - and a brand-new minion
  ID still gets auto-accepted by the real master regardless of which key
  it presents, so the mismatch never surfaced as a rejection. Both
  docker_start() and rotate's own cleanup now clear minion.pem/minion.pub
  before every container start, running as root (-u root) since the
  target can be a root-owned bind-mounted path that the image's default
  non-root user can't write to.
@prawintiru
prawintiru merged commit 46e2ff3 into main Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants