Skip to content

fix(ios): use connection rotation for frame coordinates - #4177

Open
huytdps13400 wants to merge 1 commit into
margelo:mainfrom
huytdps13400:fix/4114-ios-frame-coordinates
Open

fix(ios): use connection rotation for frame coordinates#4177
huytdps13400 wants to merge 1 commit into
margelo:mainfrom
huytdps13400:fix/4114-ios-frame-coordinates

Conversation

@huytdps13400

Copy link
Copy Markdown

Summary

  • keep the public output-relative Frame.orientation and Frame.isMirrored semantics unchanged
  • capture the physical rotation and mirroring applied to each delivered iOS video buffer
  • use those physical values for Frame ↔ Camera coordinate conversion

This replaces the closed #4175 with a smaller, data-driven fix for #4114.

Why

AVCaptureVideoPreviewLayer converts capture-device points in the camera's unrotated image space, while AVCaptureVideoDataOutput can physically rotate and mirror delivered buffers. The converter previously received the public output-relative orientation/mirroring values, so Frame and PreviewView did not necessarily refer to the same intermediate coordinate space.

The prior PR attempted to bridge the spaces by assuming every camera's native sensor orientation was .left. That is not valid across devices. This version reads the connection's actual videoRotationAngle on iOS 17+ instead. Apple explicitly documents that the default can be 180° on newer front-camera iPads and recommends accounting for the connection's default rotation. Older iOS and visionOS retain the existing videoOrientation behavior because videoRotationAngle is unavailable there.

No Android code or public API is changed.

Verification

  • existing merged Harness regression maps a square Frame region onto a square View region remains the end-to-end CI oracle
  • task-local affine regression: fixed composition ratio 1.0; old output-relative wiring 3.1604938271604937; 180° connection default and inverse round-trip pass
  • bun camera typecheck
  • bun lint-swift
  • xcodebuild -project apps/simple-camera/ios/Pods/Pods.xcodeproj -scheme VisionCamera -configuration Debug -sdk iphonesimulator -destination 'generic/platform=iOS Simulator' CODE_SIGNING_ALLOWED=NO build (arm64 + x86_64)
  • git diff --check

The physical-camera Harness run remains the final end-to-end check, especially for a Spring 2024+ front-camera iPad.

Fixes #4114.
Replaces #4175.

@vercel

vercel Bot commented Aug 30, 2026

Copy link
Copy Markdown

@huytdps13400 is attempting to deploy a commit to the Margelo Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 convertCameraPointToFramePoint ignores normalising extents on orientation swap (iOS) — square region maps to a 3.16:1 strip

1 participant