Skip to content

feat(mobile): add live preview and markup - #5161

Draft
devteapot wants to merge 6 commits into
pingdotgg:mainfrom
devteapot:agent/ipad-live-preview
Draft

feat(mobile): add live preview and markup#5161
devteapot wants to merge 6 commits into
pingdotgg:mainfrom
devteapot:agent/ipad-live-preview

Conversation

@devteapot

@devteapot devteapot commented Aug 1, 2026

Copy link
Copy Markdown

What changed

  • Adds an iPad browser pane with split and full-screen layouts, tab management, navigation controls, and direct LAN or Tailscale loading.
  • Adds an authenticated live-preview gateway for loopback desktop previews so remote mobile clients and T3 Connect can reach the same development server.
  • Adds screenshot markup with numbered callouts, regions, freehand ink, editable attachment state, and shared structured annotation context.
  • Extends desktop preview capture with viewport, page revision, element bounds, and human-review snapshots.
  • Preserves raw Vite query flags through the gateway so client bootstrap and Fast Refresh work in the iPad WebView.
  • Documents the mobile preview and markup workflow.

Why

Mobile could see preview metadata but could not reliably browse, annotate, or refresh the active desktop preview. Loopback URLs were unreachable from iPad, remote paths lacked an authenticated bridge, and URL normalization changed Vite flag queries such as ?url into ?url=, preventing the client entry from hydrating and registering JavaScript HMR modules.

Impact

Users can review the active app from iPad, work across local network, Tailscale, or T3 Connect paths, annotate a frozen frame, and send both the rendered image and source-aware context to an agent. Browser state remains isolated per mobile WebView while preview tab metadata stays synchronized through the existing server model.

Validation

  • 34 changed test files: 443 tests passed
  • Server typecheck passed
  • Mobile typecheck passed
  • Diff whitespace check passed
  • Verified the workflow on a connected iPad and desktop preview

Implemented with GPT-5.6-Sol through the Codex harness.

Note

Add live preview gateway and image markup to mobile app

  • Adds a live browser preview pane to the mobile app via a new MobileLivePreview WebView component with DOM capture, navigation controls, and a ThreadPreviewRouteScreen that renders inline or fullscreen depending on layout.
  • Introduces a server-side reverse proxy gateway (LiveGateway, LiveGatewayHttp) that issues single-use bootstrap tokens and leases to securely proxy loopback desktop preview servers to mobile clients over HTTPS.
  • Adds two new WebSocket RPC endpoints: previewReviewSnapshot (compact, bounded snapshot for review) and previewOpenLiveGateway (returns a short-lived bootstrap URL); both require the orchestration operate scope.
  • Adds image markup/annotation support: a new ImageMarkupModal lets users draw callouts on draft images; annotated attachments are flattened to PNG and stored with the original for re-editing; annotation prompts are appended to outgoing messages.
  • Extends the desktop PreviewManager with a review snapshot mode that omits accessibility trees and diagnostics, caps screenshot pixels to ~2M, and normalizes interactive elements.
  • Adds two repair migrations (036, 037) to backfill missing scopes and proof_key_thumbprint auth columns on databases that missed prior migrations.
  • Risk: live gateway leases are in-memory only; a server restart invalidates all active preview sessions.
📊 Macroscope summarized a77ca3f. 78 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a97d1380-ba70-4b5f-98e2-3cc5b17f6ad0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XXL 1,000+ changed lines (additions + deletions). labels Aug 1, 2026

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Effect service review found one convention issue: the new PreviewLiveGateway service is wired by passing its resolved instance into layer factories instead of being acquired from the Effect environment. Everything else in the changed Effect code (module namespace imports, Context.Service with an inline interface, make/layer ordering, Schema.TaggedErrorClass failures with derived messages) matches the conventions.

Posted via Macroscope — Effect Service Conventions

Comment thread apps/server/src/preview/LiveGatewayHttp.ts Outdated
Comment thread apps/server/src/ws.ts Outdated
Comment thread apps/server/src/server.ts Outdated
Comment thread apps/server/src/preview/LiveGatewayHttp.ts
Comment thread apps/mobile/src/features/preview/ThreadPreviewPane.tsx
Comment thread apps/mobile/src/features/preview/mobilePreviewDomBridge.ts
Comment thread apps/mobile/src/features/annotations/model.ts
Comment thread apps/mobile/src/features/preview/ThreadPreviewPane.tsx
Comment thread apps/server/src/preview/LiveGateway.ts Outdated
Comment thread apps/mobile/src/features/annotations/model.ts Outdated
Comment thread apps/mobile/src/features/preview/mobilePreviewDomBridge.ts Outdated
Comment thread apps/mobile/src/features/threads/ThreadFeed.tsx
Comment thread apps/mobile/src/features/preview/previewLiveTarget.ts

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One finding: a real decode failure is dropped when translating to PreviewReviewSnapshotMalformedError. Everything else in the new Effect service code (the PreviewLiveGateway tag/make/makeLive/liveLayer module, the gateway HTTP route and middleware layers, and the review-snapshot compaction) now acquires its dependencies from the environment and follows the namespace-import and error-modeling conventions.

Posted via Macroscope — Effect Service Conventions

Comment thread apps/server/src/ws.ts
Comment thread apps/mobile/src/features/annotations/model.ts

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Effect service surfaces in this PR (apps/server/src/preview/LiveGateway.ts, LiveGatewayHttp.ts, ReviewSnapshot.ts, ws.ts, server.ts, and the new mobile/client-runtime modules).

The service definition, dependency acquisition, and error modelling now look consistent with the conventions: PreviewLiveGateway is a single-file Context.Service with an inline interface, liveGatewayBootstrapRouteLayer / liveGatewayProxyLayer / websocketRpcRouteLayer acquire the tag from the environment, the canonical layer is provided once in server.ts, and PreviewReviewSnapshotMalformedError now preserves the decode cause alongside its structural fields.

Two remaining naming findings on the new gateway module are noted inline.

Posted via Macroscope — Effect Service Conventions

Comment thread apps/server/src/preview/LiveGateway.ts Outdated
return gateway;
});

export const liveLayer = Layer.effect(PreviewLiveGateway, makeLive);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module owns a single concrete implementation, so its canonical exports should be make and layer; makeLive/liveLayer is the only such pair under apps/server/src (every other service exports export const layer = Layer.effect(Tag, make)).

Suggest folding the SessionStore subscription into make and exporting export const layer = Layer.effect(PreviewLiveGateway, make), with LiveGateway.test.ts providing a test SessionStore layer. If you prefer to keep the unsupervised constructor as a test seam, renaming just liveLayerlayer (and its two call sites in server.ts / server.test.ts) also resolves this.

Posted via Macroscope — Effect Service Conventions

Comment thread apps/server/src/server.test.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant