Skip to content

Rosetta x86-64 emulation and multi-platform image pull for vz guests#279

Open
rgarcia wants to merge 25 commits into
mainfrom
hypeship/spec-rosetta-x86
Open

Rosetta x86-64 emulation and multi-platform image pull for vz guests#279
rgarcia wants to merge 25 commits into
mainfrom
hypeship/spec-rosetta-x86

Conversation

@rgarcia

@rgarcia rgarcia commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Ships Rosetta x86-64 emulation for macOS (vz) Linux guests plus
multi-platform image pull, so amd64 OCI images run inside arm64 Apple
Silicon microVMs.

Rosetta emulation (host + guest). The vz-shim attaches a
VZVirtioFileSystemDevice + VZLinuxRosettaDirectoryShare; the guest init
binary mounts it and registers a binfmt_misc handler (with the F
fix-binary flag, and a systemd binfmt.d drop-in so the rule survives a
systemd flush) so x86-64 ELFs dispatch through Rosetta transparently. Threaded
through hypervisor.VMConfig, shimconfig.ShimConfig (+ RosettaMountTag),
and vmconfig.Config. Requires Rosetta installed on the host; the shim fails
with an actionable error otherwise.

Multi-platform image pull. Image and instance creation accept a
Docker-style platform field (os/arch[/variant], e.g. linux/amd64),
modeled on docker --platform. A new images.Platform type is the single
source of truth for parsing/validation. Today only os linux is supported
and arch must be amd64 or arm64; bad values fail fast. The recorded
platform comes from the pulled image config (not the request), and a
request/manifest mismatch fails the build. A non-host platform pins the ref to
its manifest digest while preserving the user's tag, and the build pulls by
digest so the right architecture is fetched; boot-by-tag auto-pull threads the
instance platform.

Automatic Rosetta (no user flag). Following Docker Desktop, Rosetta is a
host capability, not a per-container flag. When an instance's image targets a
non-host architecture, createInstance enables Rosetta automatically on
vz/Apple-silicon hosts and rejects emulated images on any other host with an
emulation-agnostic error. There is no enable_rosetta API field —
EnableRosetta is an internal, derived field.

Changes

  • lib/images: Platform type + helpers; record manifest platform in
    buildImage; preserve tag and pull-by-digest for non-host platforms;
    Platform replaces Architecture on the image types/metadata/mirror request.
  • openapi.yaml (+ regenerated lib/oapi): platform on CreateImageRequest,
    CreateInstanceRequest, and read-only on Image.
  • cmd/api: wire platform through both handlers.
  • lib/instances: CreateInstanceRequest.Platform; derive Rosetta via a pure
    deriveEnableRosetta helper; remove the user-facing EnableRosetta.
  • cmd/test-prewarm: mirror the amd64 image under a platform-encoded tag so the
    E2E resolves an unambiguous single-platform manifest.
  • Tests: replace the injected-ELF Rosetta E2E with a real amd64-image boot test
    (TestVZRosettaImageX86); add Linux unit tests for platform
    parsing/validation, the metadata round-trip, manifest recording, and the
    auto-Rosetta derivation.
  • docs/proposals/rosetta-x86.md: updated to reflect what shipped (automatic
    Rosetta, platform field, multi-platform pull).

Test plan

  • CI test (Linux): platform + auto-Rosetta unit tests, full suite.
  • CI test-darwin (Apple M1, Rosetta installed): TestVZRosettaImageX86
    boots a real amd64 alpine image and execs an amd64 binary via Rosetta.
  • CI e2e-install.

🤖 Generated with Claude Code


Note

High Risk
Touches instance create, image resolution, and vz VM device config (Rosetta share); wrong platform or binfmt behavior could break boots or cross-arch exec on macOS.

Overview
Adds Rosetta x86-64 emulation for macOS vz Linux guests and Docker-style platform on image/instance APIs so amd64 OCI images can run on Apple Silicon microVMs.

Rosetta (automatic, no API flag). When a resolved image’s architecture differs from the host, instance create turns on internal EnableRosetta on darwin/arm64 + vz and rejects emulated images elsewhere. The flag flows through hypervisor.VMConfigshimconfigvz-shim, which attaches a Linux Rosetta virtio-fs share (configureDirectorySharing); guest init mounts it and registers binfmt_misc (with F + optional systemd binfmt.d drop-in).

Multi-platform images. New images.Platform parsing/validation (linux + amd64/arm64). Create image/instance accept optional platform; pulls resolve the right manifest, record platform from image config, pull by digest while keeping tags, and pin instance metadata to digest when platform is explicit. API returns invalid_platform where applicable; OpenAPI/oapi regenerated.

Supporting changes: ~/ path expansion in API config; macOS mkfs.ext4 lookup; prewarm mirrors alpine:3.19 at linux/amd64; make test-vz-shim-signed for entitlement-gated vz-shim tests; docs (Rosetta RFC, Rosetta install in README/DEVELOPMENT).

Reviewed by Cursor Bugbot for commit 401d07e. Bugbot is set up for automated code reviews on this repo. Configure here.

rgarcia and others added 15 commits June 7, 2026 02:56
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Thread an EnableRosetta toggle through both vz configuration channels so
Apple silicon Linux microVMs can execute x86-64 binaries via Rosetta.

Host: the vz-shim attaches a Linux Rosetta virtio-fs share when enabled.
The arm64-only binding calls live in rosetta_arm64.go with a darwin/amd64
stub in rosetta_other.go that errors if Rosetta is requested, so the shim
still builds for Intel macOS. createVM calls configureDirectorySharing,
which errors hard when Rosetta is NotInstalled/NotSupported.

Guest: init mounts the share and registers a binfmt_misc handler for
x86-64 ELF using the "F" fix-binary flag so the pinned interpreter fd
survives the later chroot into the overlay rootfs. The rule string is a
pure function (rosettaBinfmtRule) with a unit test for the exact output.

The external instance-API/CLI surface (enable_rosetta) is deferred to a
follow-up since it requires regenerating the OpenAPI types; the internal
plumbing is wired and EnableRosetta is settable for tests.
Run only the Rosetta feature tests so CI gives fast, focused signal:
- test-darwin: make test TEST='Rosetta|DirectorySharing|ShimConfig|BuildShimConfig|GuestConfig|Binfmt'
  plus a build matrix that cross-compiles cmd/vz-shim for darwin/arm64 and
  darwin/amd64 to prove the arm64 file + non-arm64 stub split.
- test (linux): narrowed to Binfmt|GuestConfig so the guest-side
  rosettaBinfmtRule test (lib/system/init is excluded from the darwin job)
  and the guest-config propagation test still run.
- e2e-install: disabled (unrelated to this feature).

Revert this commit before merge to restore the full test matrix.
Rename the configureDirectorySharing tests to TestRosetta* so every
feature test name contains "Rosetta", letting a metacharacter-free
-run regex select the whole suite.
The Makefile's test recipe interpolates -run=$(TEST) unquoted, so an
alternation like 'A|B' is split by the shell. Pass the single token
'Rosetta' instead; all feature tests are named accordingly.

Code-Hex/vz v3.7.1 does not compile for darwin/amd64 (its generated
virtualmachinestate_string.go references arm64-only state constants), so
the build matrix cross-compiles cmd/vz-shim for darwin/arm64 only.

Revert this commit before merge to restore the full test matrix.
The darwin Run tests step still passed a pipe-delimited regex, which the
Makefile's unquoted -run=$(TEST) split in the shell. Use the single
'Rosetta' token to match every feature test on the macOS runner.

Revert before merge.
…s/tests

Guest init Rosetta setup now survives systemd-mode boots and tolerates a
pre-existing handler:

- Mount the Rosetta share inside the overlay rootfs at /opt/hypeman/rosetta so
  the interpreter keeps a stable guest-absolute path across the chroot and is
  not shadowed by the /run tmpfs systemd remounts at boot.
- In systemd mode, also drop /usr/lib/binfmt.d/rosetta.conf into the rootfs so
  systemd-binfmt re-registers the handler after its boot-time status=-1 flush,
  which otherwise silently wipes the live rule when the image ships binfmt.d
  entries. The F flag pins the interpreter fd but does not survive an explicit
  flush.
- Treat an EEXIST from the binfmt register write as already-registered.

Host stub and proposal corrected to stop claiming cmd/vz-shim builds for
darwin/amd64 (Code-Hex/vz v3.7.1 is arm64-only); the !arm64 stub only satisfies
the type checker, mirroring save_restore_unsupported.go.

Split the host availability test so the attach+validate assertions live in a
test that skips visibly when Rosetta is not installed, instead of asserting
nothing on a stock runner.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add VERBOSE=1 to the darwin Run tests step so the log shows per-subtest
PASS/SKIP and records which Rosetta availability branch ran (the attach+validate
path skips when Rosetta is not installed on the runner). Tighten the amd64
build-matrix comment to state the precise reason vz v3.7.1 is arm64-only.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Boots a vz Linux guest with EnableRosetta and runs an injected static
x86-64 ELF inside it, which executes only if the guest mounted the
Rosetta share and registered the binfmt_misc handler and the host
attached the share. Skips where host Rosetta is unavailable.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
binfmt_misc dispatch is global to the kernel; the handler the guest init
registers is not visible at /proc/sys/fs/binfmt_misc inside the chrooted
container, so executing an injected x86-64 ELF is the correct end-to-end
proof of Rosetta emulation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add Architecture to CreateImageRequest and MirrorRequest. When a requested
architecture differs from the host, resolve the arch-specific manifest digest
and pin the reference to it so the existing digest-keyed pull fetches that
exact platform and caches it under a distinct digest. Persist the architecture
on image metadata and surface it on Image, defaulting to the host arch for
images predating arch tracking.

Reject instance creation when the image architecture differs from the host
kernel and EnableRosetta is not set, since such a guest cannot boot natively.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Let prewarm entries carry an optional architecture and mirror alpine:3.19 at
amd64. 3.19 is not otherwise mirrored, so its local ref is a plain amd64
manifest with no collision against the host-arch alpine entries.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
TestVZRosettaImageX86 pulls the amd64 alpine image and boots it on the arm64
vz host with EnableRosetta, so the entire guest userland is x86-64. The amd64
entrypoint only runs under Rosetta, and execing an image binary proves
emulation end to end without injecting anything.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the just-added bare image Architecture string with a Docker-style
Platform type (os/arch[/variant]) across images and instances, and finish the
multi-platform feature end to end:

- Add images.Platform with ParsePlatform/Normalize/String/ToGCR plus host and
  needsEmulation helpers as the single source of truth for platform handling.
- Surface a `platform` field on the image-create, instance-create, and image
  response schemas in openapi.yaml; wire the handlers.
- Record the platform from the pulled image config (not the request) and fail
  the build on a request/manifest mismatch.
- Preserve the user's tag when pinning a non-host platform to its manifest
  digest, and pull by the digest-pinned ref so the right architecture is
  fetched; thread the instance platform into boot-by-tag auto-pull.
- Derive Rosetta automatically when the image architecture differs from the
  host (vz/Apple silicon), replacing the user-facing enable_rosetta flag; reject
  emulated images on other hosts.
- Encode the platform into the prewarm mirror's local tag so the image E2E
  resolves an unambiguous single-platform manifest.
- Replace the injected-ELF Rosetta E2E with a real amd64-image boot test; add
  Linux unit tests for platform parsing/validation, manifest recording, and the
  auto-Rosetta derivation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@rgarcia rgarcia changed the title Add design proposal: Rosetta x86-64 emulation for vz Linux guests Rosetta x86-64 emulation and multi-platform image pull for vz guests Jun 7, 2026
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown

✱ Stainless preview builds for hypeman

This PR will update the hypeman SDKs with the following commit message.

feat: Add design proposal: Rosetta x86-64 emulation for vz Linux guests

Edit this comment to update it. It will appear in the SDK's changelogs.

hypeman-openapi studio · code · diff

Your SDK build had at least one "note" diagnostic, but this did not represent a regression.
generate ✅

hypeman-typescript studio · code · diff

Your SDK build had at least one "note" diagnostic, but this did not represent a regression.
generate ✅build ✅lint ❗test ✅

npm install https://pkg.stainless.com/s/hypeman-typescript/9115a374038b8e285cec7d9ddc10c4ba8cfb5fbd/dist.tar.gz
hypeman-go studio · code · diff

Your SDK build had at least one "note" diagnostic, but this did not represent a regression.
generate ✅build ✅lint ✅test ✅

go get github.com/stainless-sdks/hypeman-go@0f8f0ad79ec8ef619da0a5d0221b7f5dc2c338e9

This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push.
If you push custom code to the preview branch, re-run this workflow to update the comment.
Last updated: 2026-06-08 20:00:55 UTC

rgarcia and others added 3 commits June 7, 2026 17:19
Mirror the amd64 prewarm image to the plain local ref (matching what
MirrorBaseImage actually pushes) instead of a platform-encoded tag that
nothing pushed to, and remove the now-unused LocalPlatformTag. Skip the
in-process Rosetta attach validation when the unsigned test binary lacks
the virtualization entitlement; that path is covered by the E2E.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The vz-shim tests call Virtualization.framework in-process, which needs
the com.apple.security.virtualization entitlement. Compile the test
binary, ad-hoc-sign it with vz.entitlements (as the shim itself is
signed), and run it with HYPEMAN_VZ_SIGNED=1 so the Rosetta attach test
runs for real instead of skipping. cmd/vz-shim is excluded from the
plain darwin run to avoid an unsigned duplicate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Surfaces per-test PASS/SKIP for the entitlement-gated Rosetta test so the
signed run is observably real, not a silent skip.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@rgarcia rgarcia marked this pull request as ready for review June 8, 2026 02:14

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Instance create ignores platform
    • createInstance now validates explicit req.Platform, reconciles mismatched local tags by creating/waiting on the requested-platform digest, and re-fetches the image before Rosetta derivation.
  • ✅ Fixed: Prewarm cache uses host arch
    • Prewarm manifest inspection now uses platform-aware inspection (img.Platform when set) for both cache-hit checks and post-mirror digest reads, preventing host-arch false cache hits.

Create PR

Or push these changes by commenting:

@cursor push a140a12f3f
Preview (a140a12f3f)
diff --git a/cmd/test-prewarm/main.go b/cmd/test-prewarm/main.go
--- a/cmd/test-prewarm/main.go
+++ b/cmd/test-prewarm/main.go
@@ -157,7 +157,11 @@
 		return manifestImage{}, err
 	}
 
-	if digest, err := inspector.InspectManifest(ctx, localRef); err == nil {
+	inspectDigest := func() (string, error) {
+		return inspector.InspectManifestForPlatform(ctx, localRef, img.Platform)
+	}
+
+	if digest, err := inspectDigest(); err == nil {
 		return manifestImage{Source: img.Source, LocalRef: localRef, Digest: digest, CacheHit: true}, nil
 	}
 
@@ -169,7 +173,7 @@
 		return manifestImage{}, err
 	}
 
-	digest, err := inspector.InspectManifest(ctx, localRef)
+	digest, err := inspectDigest()
 	if err != nil {
 		digest = res.Digest
 	}

diff --git a/lib/images/oci_public.go b/lib/images/oci_public.go
--- a/lib/images/oci_public.go
+++ b/lib/images/oci_public.go
@@ -3,6 +3,7 @@
 import (
 	"context"
 	"fmt"
+	"strings"
 )
 
 // OCIClient is a public wrapper for system manager to use OCI operations
@@ -25,6 +26,19 @@
 	return c.client.inspectManifest(ctx, imageRef)
 }
 
+// InspectManifestForPlatform inspects a remote image to get its digest for a
+// specific platform variant (or host platform when empty).
+func (c *OCIClient) InspectManifestForPlatform(ctx context.Context, imageRef, platform string) (string, error) {
+	if strings.TrimSpace(platform) == "" {
+		return c.client.inspectManifest(ctx, imageRef)
+	}
+	resolvedPlatform, err := ParsePlatform(platform)
+	if err != nil {
+		return "", err
+	}
+	return c.client.inspectManifestWithPlatform(ctx, imageRef, resolvedPlatform.ToGCR())
+}
+
 // InspectManifestForLinux is an alias for InspectManifest (all images target Linux)
 func (c *OCIClient) InspectManifestForLinux(ctx context.Context, imageRef string) (string, error) {
 	return c.InspectManifest(ctx, imageRef)

diff --git a/lib/instances/create.go b/lib/instances/create.go
--- a/lib/instances/create.go
+++ b/lib/instances/create.go
@@ -95,6 +95,16 @@
 	imageCtx, imageSpanEnd := m.startLifecycleStep(ctx, "resolve_image",
 		attribute.String("operation", "resolve_image"),
 	)
+	waitNameForDigest := func(digest string) string {
+		if strings.TrimSpace(digest) == "" {
+			return req.Image
+		}
+		normalizedRef, parseErr := images.ParseNormalizedRef(req.Image)
+		if parseErr != nil {
+			return req.Image
+		}
+		return normalizedRef.Repository() + "@" + digest
+	}
 	imageInfo, err := m.imageManager.GetImage(imageCtx, req.Image)
 	if err != nil {
 		if err == images.ErrNotFound {
@@ -102,7 +112,7 @@
 			// background and wait up to 5 seconds for it to complete. Thread the
 			// requested platform so boot-by-tag pulls the right architecture.
 			log.InfoContext(ctx, "image not found locally, auto-pulling", "image", req.Image, "platform", req.Platform)
-			_, pullErr := m.imageManager.CreateImage(imageCtx, images.CreateImageRequest{Name: req.Image, Platform: req.Platform})
+			pulledImage, pullErr := m.imageManager.CreateImage(imageCtx, images.CreateImageRequest{Name: req.Image, Platform: req.Platform})
 			if pullErr != nil {
 				imageSpanEnd(pullErr)
 				log.ErrorContext(ctx, "failed to auto-pull image", "image", req.Image, "error", pullErr)
@@ -112,7 +122,11 @@
 			// we return an error but let it continue in the background.
 			pullCtx, pullCancel := context.WithTimeout(imageCtx, 5*time.Second)
 			defer pullCancel()
-			if waitErr := m.imageManager.WaitForReady(pullCtx, req.Image); waitErr != nil {
+			waitName := req.Image
+			if pulledImage != nil {
+				waitName = waitNameForDigest(pulledImage.Digest)
+			}
+			if waitErr := m.imageManager.WaitForReady(pullCtx, waitName); waitErr != nil {
 				imageSpanEnd(waitErr)
 				log.InfoContext(ctx, "image pull not ready within timeout, pull continues in background", "image", req.Image, "error", waitErr)
 				return nil, fmt.Errorf("%w: image %s is being pulled, please try again shortly", ErrImageNotReady, req.Image)
@@ -130,6 +144,52 @@
 			return nil, fmt.Errorf("get image: %w", err)
 		}
 	}
+	if strings.TrimSpace(req.Platform) != "" {
+		requestedPlatform, parseErr := images.ParsePlatform(req.Platform)
+		if parseErr != nil {
+			imageSpanEnd(parseErr)
+			log.ErrorContext(ctx, "invalid requested image platform", "image", req.Image, "platform", req.Platform, "error", parseErr)
+			return nil, parseErr
+		}
+		resolvedPlatform, parseErr := images.ParsePlatform(imageInfo.Platform)
+		platformMismatch := parseErr != nil || resolvedPlatform.String() != requestedPlatform.String()
+		if platformMismatch {
+			log.InfoContext(ctx,
+				"resolved image platform differs from request, reconciling",
+				"image", req.Image,
+				"requested_platform", requestedPlatform.String(),
+				"resolved_platform", imageInfo.Platform,
+			)
+			pulledImage, pullErr := m.imageManager.CreateImage(imageCtx, images.CreateImageRequest{Name: req.Image, Platform: requestedPlatform.String()})
+			if pullErr != nil {
+				imageSpanEnd(pullErr)
+				log.ErrorContext(ctx, "failed to resolve requested image platform", "image", req.Image, "platform", requestedPlatform.String(), "error", pullErr)
+				return nil, fmt.Errorf("resolve image for platform %s: %w", requestedPlatform, pullErr)
+			}
+			pullCtx, pullCancel := context.WithTimeout(imageCtx, 5*time.Second)
+			defer pullCancel()
+			waitName := req.Image
+			if pulledImage != nil {
+				waitName = waitNameForDigest(pulledImage.Digest)
+			}
+			if waitErr := m.imageManager.WaitForReady(pullCtx, waitName); waitErr != nil {
+				imageSpanEnd(waitErr)
+				log.InfoContext(ctx,
+					"platform-specific image pull not ready within timeout, pull continues in background",
+					"image", req.Image,
+					"platform", requestedPlatform.String(),
+					"error", waitErr,
+				)
+				return nil, fmt.Errorf("%w: image %s for platform %s is being pulled, please try again shortly", ErrImageNotReady, req.Image, requestedPlatform)
+			}
+			imageInfo, err = m.imageManager.GetImage(imageCtx, req.Image)
+			if err != nil {
+				imageSpanEnd(err)
+				log.ErrorContext(ctx, "failed to get image after platform reconciliation", "image", req.Image, "platform", requestedPlatform.String(), "error", err)
+				return nil, fmt.Errorf("get image after platform reconciliation: %w", err)
+			}
+		}
+	}
 	imageSpanEnd(nil)
 
 	if imageInfo.Status != images.StatusReady {

You can send follow-ups to the cloud agent here.

Comment thread lib/instances/create.go
Comment thread cmd/test-prewarm/main.go
@firetiger-agent

Copy link
Copy Markdown

Created a monitoring plan for this PR.

What this PR does: Adds a platform field (linux/amd64, linux/arm64) to the Image and Instance APIs, and introduces end-to-end Apple Rosetta support so users can run x86 containers on arm64 hosts.

Intended effect:

  • invalid_platform 400 responses: baseline 0 (new code path); confirmed if invalid platform values return 400 invalid_platform rather than 500 or a generic 400
  • Platform field in Image responses: baseline absent; confirmed if newly created cross-arch images include platform in their API response
  • Rosetta instance creation: baseline 0 (new feature); confirmed if first amd64 instance on arm64 host reaches RUNNING without any "failed to set up rosetta" or "rosetta requested but not installed" errors

Risks:

  • Instance creation regression — "failed to create instance" errors in API logs, alert if sustained >5,000/hr (baseline ~2,400–2,500/hr)
  • False-positive invalid_platform 400 — any invalid_platform error on a request that did not send a platform field (expected: 0; any occurrence is a bug)
  • Silent Rosetta binfmt failure — "failed to set up rosetta" ERROR in hypeman Railway stdout; amd64 instance would start but x86 binaries would fail silently; alert on any occurrence
  • Rosetta not installed on host — "rosetta requested but not installed" or "rosetta requested but not supported" in vz-shim logs; alert on any occurrence
  • General API error rate spike — API ERROR logs, alert if >50,000/hr (baseline ~17K–25K/hr active hours)

Status updates will be posted automatically on this PR as monitoring progresses.

View monitor

rgarcia and others added 2 commits June 8, 2026 02:48
CreateInstance resolved the image via GetImage(tag), which returns
whatever architecture the tag currently points to, so a request for a
non-cached architecture of an already-local image was silently ignored
(wrong arch booted, emulation disabled). When a platform is requested and
the resolved image's architecture differs, resolve that platform's image
by digest (which the build keys its metadata on) instead of by the tag.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Platform pull waits wrong tag
    • resolveImageForCreate now builds a digest-pinned reference from the CreateImage result and uses that ref for WaitForReady/GetImage when a platform-specific pull is still pending, preventing tag-symlink races.
  • ✅ Fixed: Binfmt EEXIST not detected
    • alreadyRegistered now treats syscall.EEXIST as benign (including wrapped os.PathError cases), so duplicate :rosetta: registrations no longer surface as hard errors.

Create PR

Or push these changes by commenting:

@cursor push 12bc1f0435
Preview (12bc1f0435)
diff --git a/lib/instances/create.go b/lib/instances/create.go
--- a/lib/instances/create.go
+++ b/lib/instances/create.go
@@ -608,11 +608,18 @@
 		if err != nil {
 			return nil, fmt.Errorf("resolve image %s for platform %s: %w", imageName, platform, err)
 		}
+		resolvedName := img.Name
+		if img.Digest != "" {
+			resolvedName, err = imageDigestRef(img.Name, img.Digest)
+			if err != nil {
+				return nil, fmt.Errorf("build resolved image ref: %w", err)
+			}
+		}
 		if img.Status != images.StatusReady {
-			if err := waitForImagePull(ctx, imageManager, img.Name, log); err != nil {
+			if err := waitForImagePull(ctx, imageManager, resolvedName, log); err != nil {
 				return nil, err
 			}
-			img, err = imageManager.GetImage(ctx, img.Name)
+			img, err = imageManager.GetImage(ctx, resolvedName)
 			if err != nil {
 				return nil, fmt.Errorf("get image after platform resolve: %w", err)
 			}
@@ -648,6 +655,14 @@
 	return img, nil
 }
 
+func imageDigestRef(imageName, digest string) (string, error) {
+	ref, err := images.ParseNormalizedRef(imageName)
+	if err != nil {
+		return "", fmt.Errorf("parse image ref %q: %w", imageName, err)
+	}
+	return ref.Repository() + "@" + digest, nil
+}
+
 func waitForImagePull(ctx context.Context, imageManager createImageResolver, imageName string, log *slog.Logger) error {
 	pullCtx, pullCancel := context.WithTimeout(ctx, 5*time.Second)
 	defer pullCancel()

diff --git a/lib/instances/create_image_test.go b/lib/instances/create_image_test.go
--- a/lib/instances/create_image_test.go
+++ b/lib/instances/create_image_test.go
@@ -95,6 +95,67 @@
 	}
 }
 
+func TestResolveImageForCreateWithPlatformWaitsOnResolvedDigest(t *testing.T) {
+	t.Parallel()
+
+	const digest = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
+	const tagRef = "docker.io/library/alpine:3.19"
+	const digestRef = "docker.io/library/alpine@" + digest
+
+	waitName := ""
+	getNames := make([]string, 0, 2)
+	resolver := createImageResolverFake{
+		getImage: func(_ context.Context, name string) (*images.Image, error) {
+			getNames = append(getNames, name)
+			switch name {
+			case digestRef:
+				return &images.Image{
+					Name:     tagRef,
+					Digest:   digest,
+					Platform: "linux/amd64",
+					Status:   images.StatusReady,
+				}, nil
+			case tagRef:
+				return &images.Image{
+					Name:     tagRef,
+					Digest:   "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+					Platform: "linux/arm64",
+					Status:   images.StatusReady,
+				}, nil
+			default:
+				t.Fatalf("unexpected GetImage name %q", name)
+				return nil, nil
+			}
+		},
+		createImage: func(_ context.Context, req images.CreateImageRequest) (*images.Image, error) {
+			return &images.Image{
+				Name:     req.Name,
+				Digest:   digest,
+				Platform: "linux/amd64",
+				Status:   images.StatusPending,
+			}, nil
+		},
+		waitReady: func(_ context.Context, name string) error {
+			waitName = name
+			return nil
+		},
+	}
+
+	img, err := resolveImageForCreate(context.Background(), resolver, tagRef, "linux/amd64", slog.Default())
+	if err != nil {
+		t.Fatalf("resolve image: %v", err)
+	}
+	if waitName != digestRef {
+		t.Fatalf("expected WaitForReady on %q, got %q", digestRef, waitName)
+	}
+	if len(getNames) != 1 || getNames[0] != digestRef {
+		t.Fatalf("expected GetImage on %q, got %#v", digestRef, getNames)
+	}
+	if img.Platform != "linux/amd64" {
+		t.Fatalf("expected amd64 image, got %s", img.Platform)
+	}
+}
+
 func TestResolveImageForCreateWithoutPlatformUsesExistingImage(t *testing.T) {
 	t.Parallel()
 

diff --git a/lib/system/init/rosetta.go b/lib/system/init/rosetta.go
--- a/lib/system/init/rosetta.go
+++ b/lib/system/init/rosetta.go
@@ -107,7 +107,7 @@
 // alreadyRegistered reports whether a binfmt_misc register write failed only
 // because a handler of that name already exists (EEXIST).
 func alreadyRegistered(err error) bool {
-	return errors.Is(err, fs.ErrExist)
+	return errors.Is(err, fs.ErrExist) || errors.Is(err, syscall.EEXIST)
 }
 
 // writeBinfmtdConf drops a systemd binfmt.d config that re-registers Rosetta

diff --git a/lib/system/init/rosetta_test.go b/lib/system/init/rosetta_test.go
--- a/lib/system/init/rosetta_test.go
+++ b/lib/system/init/rosetta_test.go
@@ -3,6 +3,7 @@
 import (
 	"errors"
 	"io/fs"
+	"os"
 	"strings"
 	"syscall"
 	"testing"
@@ -37,6 +38,7 @@
 	// EEXIST means a :rosetta: handler is already registered; treat it as success.
 	assert.True(t, alreadyRegistered(syscall.EEXIST))
 	assert.True(t, alreadyRegistered(fs.ErrExist))
+	assert.True(t, alreadyRegistered(&os.PathError{Op: "write", Path: binfmtRegister, Err: syscall.EEXIST}))
 	// Any other error (or none) is not an "already registered" condition.
 	assert.False(t, alreadyRegistered(nil))
 	assert.False(t, alreadyRegistered(syscall.EACCES))

You can send follow-ups to the cloud agent here.

Comment thread lib/instances/create.go Outdated
Comment thread lib/system/init/rosetta.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Committed scratch TODO file
    • Deleted the root TODO.md scratch note file that was accidentally committed.

Create PR

Or push these changes by commenting:

@cursor push 8662dfd780
Preview (8662dfd780)
diff --git a/TODO.md b/TODO.md
deleted file mode 100644
--- a/TODO.md
+++ /dev/null
@@ -1,3 +1,0 @@
-TODO:
-- respect ~/ paths in data_dir and other config paths
-- update hypeman-cli for --platform from pr 279
\ No newline at end of file

You can send follow-ups to the cloud agent here.

Comment thread TODO.md

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Start resolves image tag not platform
    • Instance create now persists a digest-pinned image reference and start/restore resolve images from that pinned ref (with fallback for older metadata), preventing tag-based platform drift on restart.

Create PR

Or push these changes by commenting:

@cursor push 28c4f4a867
Preview (28c4f4a867)
diff --git a/lib/instances/create.go b/lib/instances/create.go
--- a/lib/instances/create.go
+++ b/lib/instances/create.go
@@ -314,10 +314,16 @@
 	}
 
 	// 11. Create instance metadata
+	resolvedImageRef, err := pinnedImageName(req.Image, imageInfo.Digest)
+	if err != nil {
+		log.ErrorContext(ctx, "failed to persist resolved image reference", "image", req.Image, "digest", imageInfo.Digest, "error", err)
+		return nil, err
+	}
 	stored := &StoredMetadata{
 		Id:                       id,
 		Name:                     req.Name,
 		Image:                    req.Image,
+		ResolvedImage:            resolvedImageRef,
 		Size:                     size,
 		HotplugSize:              hotplugSize,
 		OverlaySize:              overlaySize,

diff --git a/lib/instances/restore.go b/lib/instances/restore.go
--- a/lib/instances/restore.go
+++ b/lib/instances/restore.go
@@ -233,9 +233,10 @@
 			releaseNetwork()
 			return nil, fmt.Errorf("configure egress proxy: %w", err)
 		}
-		imageInfo, err := m.imageManager.GetImage(ctx, stored.Image)
+		imageRef := stored.RuntimeImageRef()
+		imageInfo, err := m.imageManager.GetImage(ctx, imageRef)
 		if err != nil {
-			log.ErrorContext(ctx, "failed to load image for config disk refresh", "instance_id", id, "image", stored.Image, "error", err)
+			log.ErrorContext(ctx, "failed to load image for config disk refresh", "instance_id", id, "image", imageRef, "error", err)
 			releaseNetwork()
 			return nil, fmt.Errorf("get image for restore config disk: %w", err)
 		}

diff --git a/lib/instances/start.go b/lib/instances/start.go
--- a/lib/instances/start.go
+++ b/lib/instances/start.go
@@ -80,16 +80,17 @@
 	}
 
 	// 3. Get image info (needed for buildHypervisorConfig)
-	log.DebugContext(ctx, "getting image info", "instance_id", id, "image", stored.Image)
+	imageRef := stored.RuntimeImageRef()
+	log.DebugContext(ctx, "getting image info", "instance_id", id, "image", imageRef)
 	imageCtx, imageSpanEnd := m.startLifecycleStep(ctx, "resolve_image",
 		attribute.String("instance_id", id),
 		attribute.String("hypervisor", string(stored.HypervisorType)),
 		attribute.String("operation", "resolve_image"),
 	)
-	imageInfo, err := m.imageManager.GetImage(imageCtx, stored.Image)
+	imageInfo, err := m.imageManager.GetImage(imageCtx, imageRef)
 	imageSpanEnd(err)
 	if err != nil {
-		log.ErrorContext(ctx, "failed to get image", "instance_id", id, "image", stored.Image, "error", err)
+		log.ErrorContext(ctx, "failed to get image", "instance_id", id, "image", imageRef, "error", err)
 		return nil, fmt.Errorf("get image: %w", err)
 	}
 

diff --git a/lib/instances/types.go b/lib/instances/types.go
--- a/lib/instances/types.go
+++ b/lib/instances/types.go
@@ -75,9 +75,10 @@
 // StoredMetadata represents instance metadata that is persisted to disk
 type StoredMetadata struct {
 	// Identification
-	Id    string // Auto-generated CUID2
-	Name  string
-	Image string // OCI reference
+	Id            string // Auto-generated CUID2
+	Name          string
+	Image         string // OCI reference requested at create time
+	ResolvedImage string // Digest-pinned OCI reference used for boots/restarts
 
 	// Resources (matching Cloud Hypervisor terminology)
 	Size                     int64 // Base memory in bytes
@@ -183,6 +184,18 @@
 	Phases phasetracking.Tracker `json:"phases,omitempty"`
 }
 
+// RuntimeImageRef returns the digest-pinned image reference when available.
+// Older metadata may not have ResolvedImage persisted, so we fall back to Image.
+func (m *StoredMetadata) RuntimeImageRef() string {
+	if m == nil {
+		return ""
+	}
+	if m.ResolvedImage != "" {
+		return m.ResolvedImage
+	}
+	return m.Image
+}
+
 // Instance represents a virtual machine instance with derived runtime state
 type Instance struct {
 	StoredMetadata

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 8a58eb8. Configure here.

Comment thread lib/instances/create.go
@rgarcia rgarcia requested a review from hiroTamada June 8, 2026 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant