feat: QEMU ExporterSet sidecar pattern with jumpstarter-exec (PITCREW-487) by mangelajo · Pull Request #910 · jumpstarter-dev/jumpstarter

mangelajo · 2026-07-22T14:32:56Z

Summary

Implements the QEMU ExporterSet sidecar pattern for JEP-0014, enabling the jumpstarter exporter to remotely execute commands on a co-located QEMU runtime container via a shared emptyDir volume.

jumpstarter-exec Rust binary (rust/jumpstarter-exec/): A lightweight command execution bridge using NDJSON over Unix sockets. Runs in serve mode on the runtime container (launched from the shared volume) and exec mode from the exporter sidecar, transparently proxying stdin/stdout/stderr and exit codes. Embeds a Git-derived version via build.rs / GIT_VERSION.
Cargo workspace (rust/Cargo.toml): Sets up the Rust monorepo root aligned with the rust-core-rewrite branch structure.
Container images: UBI 10 multi-stage Rust builder added to python/Containerfile; Containerfile.qemu-runtime updated with a wait-for-binary.sh entrypoint that polls for the binary on the shared volume.
Pod rendering (qemu.go): Init container copies jumpstarter-exec to the shared volume and sets JUMPSTARTER_LAUNCHER_SOCKET env var; uses IfNotPresent pull policy for local kind iteration.
QEMU driver (driver.py): Sidecar mode via launcher_socket config -- wraps commands with jumpstarter-exec exec, adjusts paths to /shared/, uses Unix socket for serial console.
Deployment: Manual deployment YAML into jumpstarter-lab (same namespace as controller/router), x86_64 KVM sample + kind-friendly no-KVM sample, and Makefile targets for image builds and controller deployment.

Test plan

make test-rust passes all tests (protocol serde, server integration, e2e binary)
make fmt and make lint pass in rust/
Build exporter / qemu-runtime / exporter-set-controller images locally and load into kind
Deploy into jumpstarter-lab with make -C controller deploy-exporterset deploy-exporterset-sample-kind
Manually validated sidecar handoff: binary copy, jumpstarter-exec serve as PID 1, remote exec of qemu-system-x86_64 --version from exporter container
Verify QEMU VM boots and serial console is accessible via the exporter (blocked on reconciler Pod creation)
CI workflows (rust-tests.yaml, lint.yaml) run successfully on this PR

Relates to: PITCREW-487

coderabbitai · 2026-07-22T14:33:06Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d6064fa-b394-4d27-a741-6c1fb12619d3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch pitcrew-487-qemu-exporterset-sidecar

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-07-22T15:12:10Z

Container Images

The following container images have been built for this PR:

Image	URI
`jumpstarter`	`quay.io/jumpstarter-dev/jumpstarter:pr-910`

Images expire after 7 days.

Implements the jmp-exec bridge for the ExporterSet sidecar pattern (JEP-0014). The binary has two modes: - serve: listens on a Unix socket and executes commands on behalf of clients, bridging stdin/stdout/stderr over NDJSON protocol - exec: connects to a serve instance and acts as a transparent subprocess proxy, suitable as a drop-in for subprocess.Popen Built with UBI 10 + rust-toolset (Red Hat-supported toolchain) and dynamically linked against glibc for runtime compatibility with UBI and Fedora containers. Also adds the Rust builder stage to python/Containerfile so the binary is included in the jumpstarter exporter image. Co-authored-by: Cursor <cursoragent@cursor.com>

The qemu-runtime container now waits for jmp-exec to be copied to /shared/ by the exporter init container, then execs it in serve mode. This enables the sidecar pattern where the exporter (init container) remotely executes QEMU commands in the runtime container via jmp-exec over a Unix socket on the shared emptyDir volume. Co-authored-by: Cursor <cursoragent@cursor.com>

The exporter init container now copies jmp-exec from the exporter image to /shared/ on startup, making it available to the runtime container. Sets JUMPSTARTER_LAUNCHER_SOCKET env var so the QEMU driver knows where to connect for remote command execution. The runtime container relies on its Containerfile ENTRYPOINT/CMD (wait-for-binary.sh) to pick up jmp-exec and start serve mode. Co-authored-by: Cursor <cursoragent@cursor.com>

When launcher_socket is configured, the QEMU driver operates in sidecar mode for the ExporterSet pattern: - Commands (qemu-system, qemu-img) are wrapped with jmp-exec exec for remote execution in the runtime container - Sockets (QMP, VNC, serial) use /shared/ paths accessible from both containers via the shared emptyDir volume - Serial console uses a Unix socket instead of PTY, with UnixNetwork driver instead of PySerial - Partition images are stored on /shared/ for cross-container access When launcher_socket is not set, the driver operates in its existing local mode with no behavioral changes. Co-authored-by: Cursor <cursoragent@cursor.com>

…64 samples Provides manifests for manually deploying the ExporterSet controller with the QEMU provisioner, independent of the operator: - deploy-exporterset-qemu.yaml: Namespace, ServiceAccount, RBAC, and Deployment for the exporter-set-controller - sample-x86_64.yaml: VirtualTargetClass and ExporterSet for x86_64 QEMU targets with KVM acceleration, configured with the launcher_socket for sidecar mode Co-authored-by: Cursor <cursoragent@cursor.com>

New targets for the ExporterSet sidecar development workflow: - docker-build-qemu-runtime: Build the qemu-runtime container image - docker-build-exporter: Build the jumpstarter exporter image (with jmp-exec) - deploy-exporterset: Install CRDs + deploy the controller (builds images unless SKIP_BUILD=1) - deploy-exporterset-sample: Apply x86_64 VirtualTargetClass and ExporterSet - undeploy-exporterset: Remove the manually deployed controller and samples Co-authored-by: Cursor <cursoragent@cursor.com>

Rename and relocate the Rust binary to rust/jumpstarter-exec/ to establish the rust/ directory as the home for Rust components and align the binary name with the project naming conventions. Updates all references across Containerfiles, Go code, Python driver, Makefile, and shell scripts. Co-authored-by: Cursor <cursoragent@cursor.com>

Create rust/Cargo.toml as a workspace root with jumpstarter-exec as its first member. Shared metadata (edition, license, repository) and dependency versions are inherited from [workspace.package] and [workspace.dependencies], matching the structure on the rust-core-rewrite branch. Co-authored-by: Cursor <cursoragent@cursor.com>

Add protocol unit tests (serde roundtrips) and integration tests covering the full serve+exec pipeline over Unix sockets: stdout/stderr forwarding, exit code propagation, stdin piping, signal delivery, concurrent connections, large output, and error handling. Add rust/Makefile with build/test/clean/lint/fmt targets, wire it into the top-level Makefile, and add GitHub Actions workflows for Rust tests and linting (clippy + rustfmt) gated on rust/** path changes. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

The Cargo workspace version is a static "0.0.0" placeholder (no crates.io publishing yet), so jumpstarter-exec previously reported a meaningless version. Add a build.rs that reads GIT_VERSION from the environment at compile time and embeds it via a version command, mirroring how the Go controller (ldflags) and Python packages (hatch-vcs) derive their version from git describe. Forward GIT_VERSION into the rust-builder stage of python/Containerfile -- CI already passes it as a build-arg for hatch-vcs, so this is picked up automatically. Also pass it from the local docker-build-exporter Makefile target. Co-authored-by: Cursor <cursoragent@cursor.com>

…ting Align the manual ExporterSet deployment with the operator/e2e namespace (jumpstarter-lab), add IfNotPresent pull policies for local images, and ship a kind-friendly sample without KVM device requirements. Co-authored-by: Cursor <cursoragent@cursor.com>

Emit structured JSON logs (level/ts/msg/component) aligned with Bella's observability work. Support --debug / JUMPSTARTER_EXEC_DEBUG for argv and I/O previews, and --log-field / JUMPSTARTER_EXEC_LOG_FIELDS for persistent correlation context on every line. Co-authored-by: Cursor <cursoragent@cursor.com>

Pass the Exporter CR into RenderPod and set JUMPSTARTER_EXEC_LOG_FIELDS (component, exporter, namespace) on the target-runtime container so jumpstarter-exec logs correlate with JEP-0013 fields. Co-authored-by: Cursor <cursoragent@cursor.com>

The operator now deploys the exporter-set-controller (#908), so remove the standalone YAML and Makefile targets. Keep sample CR manifests for local kind testing after operator install. Co-authored-by: Cursor <cursoragent@cursor.com>

Split binary copy out of the exporter native sidecar into a regular init container that exits after cp, so the exporter can stay on sleep infinity until real startup wiring is ready. Co-authored-by: Cursor <cursoragent@cursor.com>

mangelajo added the build-pr-images/jumpstarter request to build only the jumpstarter image from PR label Jul 22, 2026

mangelajo force-pushed the pitcrew-487-qemu-exporterset-sidecar branch 2 times, most recently from b9334f7 to a488a6f Compare July 24, 2026 11:01

mangelajo and others added 15 commits July 24, 2026 11:43

fix: format qemu.go with gofmt to pass CI lint

84e9ff4

Co-authored-by: Cursor <cursoragent@cursor.com>

mangelajo force-pushed the pitcrew-487-qemu-exporterset-sidecar branch from e458d90 to 809ed80 Compare July 24, 2026 11:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: QEMU ExporterSet sidecar pattern with jumpstarter-exec (PITCREW-487)#910

feat: QEMU ExporterSet sidecar pattern with jumpstarter-exec (PITCREW-487)#910
mangelajo wants to merge 16 commits into
mainfrom
pitcrew-487-qemu-exporterset-sidecar

mangelajo commented Jul 22, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 22, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions Bot commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

mangelajo commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

coderabbitai Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented Jul 22, 2026

Container Images

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mangelajo commented Jul 22, 2026 •

edited

Loading

coderabbitai Bot commented Jul 22, 2026 •

edited

Loading