diff --git a/docs/adr/0001-provider-first-integration-scenarios.md b/docs/adr/0001-provider-first-integration-scenarios.md index ab2f58666c..ad71b6bc9b 100644 --- a/docs/adr/0001-provider-first-integration-scenarios.md +++ b/docs/adr/0001-provider-first-integration-scenarios.md @@ -4,69 +4,226 @@ Accepted -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Provider-first scenario -> coverage and semantic provider operations remain accepted. The monolithic `Interactor` and the -> request callback-stack topology are superseded one command at a time by request-bound runtime -> facets and explicit provider ownership. +Completed process history is in git history. + +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Provider-first scenario coverage and semantic provider operations remain accepted. +> Request-bound runtime facets and explicit provider ownership supersede the monolithic `Interactor` and the request callback-stack topology one command at a time. + +## Rules at a glance + +- For commands not yet migrated under ADR 0019, keep `Interactor` as the legacy semantic interface. +- For new command cutovers, use the bound runtime facets of ADR 0019. +- Provider-backed integration tests run the real daemon request path and replace only the providers. +- By default, provider transcripts match calls as an unordered contract. + Use ordered transcripts only when order is the behavior under test. +- Keep HTTP coverage as a narrow contract suite for JSON-RPC transport, auth, and response finalization. +- Synchronous host-tool calls are intentionally outside the provider seam. + Before a remote or cloud provider can own the path of a synchronous Apple helper, you must convert the helper. +- Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenario coverage exists. ## Context -The test suite had many mocked daemon handler and dispatch unit tests. Those tests were expensive to maintain and skipped important behavior across request admission, locking, session state, handler routing, dispatch, Interactor resolution, and platform module command translation. +The test suite had many mocked unit tests for daemon handlers and dispatch. +These tests were expensive to maintain, and they skipped important behavior in these areas: + +- Request admission. +- Locking. +- Session state. +- Handler routing. +- Dispatch. +- Interactor resolution. +- Command translation in platform modules. -Android already had an ADB provider seam. Apple-family and Linux platform modules mostly called host tools or runner commands directly, which made device-free integration tests difficult without mocking too high in the stack. +Android already had an ADB provider seam. +Apple-family and Linux platform modules mostly called host tools or runner commands directly. +Thus, device-free integration tests were difficult unless the tests mocked too high in the stack. ## Decision -For commands not yet migrated under ADR 0019, keep `Interactor` as the legacy semantic interface -between dispatch and platform behavior. New command cutovers use ADR 0019's bound runtime facets. +For commands not yet migrated under ADR 0019, keep `Interactor` as the legacy semantic interface between dispatch and platform behavior. +New command cutovers use the bound runtime facets of ADR 0019. -For commands still on the legacy adapter, use these request-scoped provider seams below platform -modules: +For commands that still use the legacy adapter, use these request-scoped provider seams below platform modules: -- Android ADB provider -- Apple tool provider with semantic `simctl`, `devicectl`, macOS helper, and macOS host subproviders -- Apple runner provider -- Linux tool provider, with semantic desktop lifecycle, accessibility, clipboard, and screenshot subproviders -- App-log provider for semantic app log stream start/stop -- Recording provider for semantic local/remote screen-recording start operations that would otherwise be raw host processes +- Android ADB provider. +- Apple tool provider with semantic `simctl`, `devicectl`, macOS helper, and macOS host subproviders. +- Apple runner provider. +- Linux tool provider with semantic subproviders for desktop lifecycle, accessibility, clipboard, and screenshots. +- App-log provider for semantic start and stop of app log streams. +- Recording provider for semantic local and remote screen-recording start operations that would otherwise be raw host processes. -Provider contracts should expose semantic operations when the platform intent is stable enough to name. Android already does this for install, pull, and port-reverse behavior. Apple tool execution has started moving in that direction with semantic `simctl`, `devicectl`, macOS helper, and macOS host runners; generic command execution remains as a local compatibility fallback for host-tool paths that have not been classified yet. The macOS host provider owns app opening, app inventory, clipboard, and appearance because remote desktop providers should not need to infer those intents from `open`, `find`, `plutil`, `pbcopy`, `pbpaste`, or `osascript`. Linux desktop lifecycle exposes semantic `openTarget` and `closeApp` operations because provider-backed integration covers those user workflows and remote desktop providers should not need to infer intent from `xdg-open`, binary launch, `wmctrl`, or `pkill`. Linux accessibility snapshots, clipboard, screenshots, and input synthesis are also semantic because the stable contracts are "capture this surface", "read/write clipboard text", "write a screenshot artifact", and "perform this user input", not the local implementation details `python3`/AT-SPI, `xclip`, `scrot`, `grim`, `xdotool`, or `ydotool`. App-log stream start/stop is semantic observability behavior rather than generic host process execution, so provider-backed integration injects an app-log provider instead of spawning `log stream` or `logcat`. iOS simulator screen recording now uses a recording provider because the stable contract is "start simulator recording for this device and output path", not the local implementation detail `xcrun simctl io recordVideo`. +When the platform intent is stable enough to name, provider contracts should expose semantic operations. +Android already does this for install, pull, and port-reverse behavior. +Apple tool execution has started to move in that direction with semantic `simctl`, `devicectl`, macOS helper, and macOS host runners. +Generic command execution remains as a local compatibility fallback for host-tool paths that are not classified yet. -Provider-backed integration tests run the real daemon request path and replace only those providers. Tests may use provider transcripts for platform command contracts and scenario transcripts for broader user workflows. Provider transcripts match calls as an unordered contract by default; use ordered transcripts only when ordering is the behavior under test. +The macOS host provider owns these operations: -Prefer an in-process provider-backed integration harness for broad scenarios: it should invoke the daemon request handler directly, preserving admission, locking, session state, handler routing, dispatch, platform modules, and provider seams without binding a TCP listener. Keep HTTP coverage as a narrow contract suite for JSON-RPC transport, auth, and response finalization. +- App open. +- App inventory. +- Clipboard. +- Appearance. -Request provider scoping is descriptor-driven inside the request router layer. A provider entry declares how it applies to the resolved device, how it installs its scoped provider, and what request-local data it exposes to handlers. Apple runner providers are request-id scoped when a request id exists; fallback by device alone is only allowed for unscoped local runner calls. +The reason is that remote desktop providers should not need to infer these intents from `open`, `find`, `plutil`, `pbcopy`, `pbpaste`, or `osascript`. -Synchronous host-tool calls are intentionally not part of the provider seam. Any remaining sync Apple helper is local-only and must be converted before a remote/cloud provider can own that path. +Linux desktop lifecycle exposes semantic `openTarget` and `closeApp` operations for two reasons: -Remaining generic Apple host-tool calls are intentionally local-only unless a new adapter creates pressure to promote them: perf process discovery/sampling (`mdfind`, `ps`), launch diagnostics (`plutil`, `otool`), local runner product build/signing (`swift`, `codesign`, sync `plutil`), local simulator UI launch (`open -a Simulator`), plist compatibility fallbacks, and fallback implementations behind semantic macOS providers. Provider-backed integration scenarios should avoid scripting those host commands for user workflows; a user-facing workflow that depends on one of them is naming pressure for a semantic provider method. +- Provider-backed integration covers these user workflows. +- Remote desktop providers should not need to infer intent from `xdg-open`, binary launch, `wmctrl`, or `pkill`. -## Alternatives Considered +Linux accessibility snapshots, clipboard, screenshots, and input synthesis are also semantic, because their stable contracts are these: -- Mock handlers or `dispatchCommand`: cheaper to write, but it skips request admission, locking, session state, and platform command translation, which were the main sources of test blind spots. -- Put the seam at `Interactor`: simpler and more uniform, but it bypasses platform modules and would not catch the iOS/Linux host-tool wiring issues that motivated this change. -- Start with a full semantic provider per platform operation: cleaner end state, but too much surface to name correctly in one pass. The migration starts where contracts already exist or where tests create pressure. -- Run every provider-backed integration scenario through HTTP: maximum end-to-end coverage, but it makes most scenarios pay for TCP setup, sandbox permissions, and transport timeouts even when transport behavior is not under test. +- "capture this surface" +- "read/write clipboard text" +- "write a screenshot artifact" +- "perform this user input" -## Consequences +The local implementation details `python3`/AT-SPI, `xclip`, `scrot`, `grim`, `xdotool`, and `ydotool` are not the contracts. + +App-log stream start/stop is semantic observability behavior, not generic host process execution. +Thus, provider-backed integration injects an app-log provider, and it does not spawn `log stream` or `logcat`. + +iOS simulator screen recording now uses a recording provider, because the stable contract is "start simulator recording for this device and output path". +The local implementation detail `xcrun simctl io recordVideo` is not the contract. -Platform command translation remains covered by integration tests without requiring real devices. +Provider-backed integration tests run the real daemon request path and replace only those providers. +Tests may use provider transcripts for platform command contracts. +Tests may use scenario transcripts for broader user workflows. +By default, provider transcripts match calls as an unordered contract. +Use ordered transcripts only when order is the behavior under test. -The request router owns a provider registry seam, but platform-specific provider applicability remains localized in that registry. The registry composes provider scopes linearly so adding a platform does not require another nested wrapper chain. +For broad scenarios, prefer an in-process provider-backed integration harness. +The harness should invoke the daemon request handler directly, without a bound TCP listener. +It should preserve: -New remote or cloud-backed adapters can implement neutral provider contracts without changing daemon, dispatch, or session contracts. Generic tool-provider fallbacks are an interim compatibility layer, not the target contract for cloud adapters. When a provider-backed integration scenario still scripts raw host commands, that is a signal to reassess whether the platform intent has become stable enough for a semantic provider method. +- Admission. +- Locking. +- Session state. +- Handler routing. +- Dispatch. +- Platform modules. +- Provider seams. -Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenario coverage exists. Unit tests remain appropriate for pure logic, parser matrices, selector matching, capability maps, and edge/error cases that integration tests would express poorly. +Keep HTTP coverage as a narrow contract suite for JSON-RPC transport, auth, and response finalization. -The trade-off is coarser failure localization: a provider-backed integration scenario catches more of the real request path but may require more diagnosis than a narrow unit test. Scenario names and provider transcript entries should stay rooted in user workflows and real e2e examples so failures remain actionable. +Inside the request router layer, descriptors drive the request provider scope. +Each provider entry declares: -Coverage is expected to improve over the old handler-heavy unit suite, but the first migration does not meet the original 90% target. The current coverage denominator also excludes some entrypoint and configuration files, so coverage should be treated as a trend signal rather than proof that every public surface is exercised. +- How the provider applies to the resolved device. +- How the provider installs its scoped provider. +- Which request-local data the provider exposes to handlers. -Operational metrics are generated by `pnpm test:integration:progress`. CI runs `pnpm test:integration:progress:check` after integration tests so public-command coverage, device-observable workflow flag coverage, and public-flag classification cannot silently regress. The script is the source of truth for provider-backed integration size, handler-unit size, mock-heavy handler pressure, public-command coverage, command-family ownership, device-observable workflow flag coverage, provider transcript pressure, and low-coverage files after a coverage run. Config, remote transport, Metro preparation, parser/client-only, report-writing, and boot-fallback flags stay in their owning unit/CLI suites and are reported as explicit exclusions rather than silently missing from the denominator. Provider pressure separates semantic Apple `simctl`/`devicectl`, macOS helper, and macOS host usage from generic Apple host-tool usage, and separates semantic Linux desktop/accessibility/clipboard/screenshot/input usage from generic Linux tool usage, so remote-adapter pressure is visible without treating named subproviders as raw shell intent. +When a request id exists, Apple runner providers are request-id scoped. +A fallback by device alone is allowed only for unscoped local runner calls. -The progress CLI should stay a thin report and check runner over a progress model. The model owns discovery, coverage classification, provider-pressure accounting, and check-failure derivation; the CLI owns Markdown output and process exit behavior. The progress script runs as a Node type-stripped TypeScript script and should consume command metadata directly when the metadata API exposes the needed facet. Source parsing inside the model is an implementation detail, not the desired long-term interface, and should remain limited to facets that are not yet represented as runtime metadata, such as mapping typed client method calls back to command names. +Synchronous host-tool calls are intentionally not part of the provider seam. +Any remaining synchronous Apple helper is local-only. +Before a remote or cloud provider can own the path of such a helper, you must convert the helper. -Every public command should have at least one provider-backed integration scenario that runs through the request router and request-scoped provider seams. Unit tests remain for parser matrices, selector matching, capability maps, malformed inputs, state machines, cleanup behavior, provider scope routing, and platform error boundaries. +Remaining generic Apple host-tool calls intentionally stay local-only, unless a new adapter creates pressure to promote them: + +- Perf process discovery and sampling: `mdfind`, `ps`. +- Launch diagnostics: `plutil`, `otool`. +- Local runner product build and signing: `swift`, `codesign`, synchronous `plutil`. +- Local simulator UI launch: `open -a Simulator`. +- Plist compatibility fallbacks. +- Fallback implementations behind semantic macOS providers. + +Provider-backed integration scenarios should avoid scripted calls to these host commands for user workflows. +A user-facing workflow that depends on one of these commands is naming pressure for a semantic provider method. + +## Alternatives Considered + +- **Mock handlers or `dispatchCommand`.** These tests are cheaper to write. + But they skip request admission, locking, session state, and platform command translation. + These areas were the main sources of test blind spots. +- **Put the seam at `Interactor`.** This is simpler and more uniform. + But it bypasses platform modules, and it would not catch the iOS/Linux host-tool wiring issues that motivated this change. +- **Start with a full semantic provider per platform operation.** This gives a cleaner end state. + But it has too much surface to name correctly in one pass. + The migration starts where contracts already exist or where tests create pressure. +- **Run every provider-backed integration scenario through HTTP.** This gives maximum end-to-end coverage. + But most scenarios would pay for TCP setup, sandbox permissions, and transport timeouts, even when transport behavior is not under test. + +## Consequences -The temporary migration roadmap is complete and intentionally removed from the repository. Future work should be justified from live pressure in the progress script, failing tests, or concrete adapter needs rather than from a standing refactor backlog. +Integration tests continue to cover platform command translation without real devices. + +The request router owns a provider registry seam. +But platform-specific provider applicability remains localized in that registry. +The registry composes provider scopes linearly. +Thus, a new platform does not require another chain of nested wrappers. + +New remote or cloud-backed adapters can implement neutral provider contracts without changes to daemon, dispatch, or session contracts. +Generic tool-provider fallbacks are an interim compatibility layer. +They are not the target contract for cloud adapters. +If a provider-backed integration scenario still scripts raw host commands, that is a signal to reassess whether the platform intent is stable enough for a semantic provider method. + +Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenario coverage exists. +Unit tests remain appropriate for pure logic, parser matrices, selector matching, capability maps, and edge/error cases that integration tests would express poorly. + +The trade-off is coarser failure localization. +A provider-backed integration scenario catches more of the real request path. +But its failures may require more diagnosis than failures in a narrow unit test. +Scenario names and provider transcript entries should stay rooted in user workflows and real e2e examples, so that failures remain actionable. + +Coverage is expected to improve over the old handler-heavy unit suite. +But the first migration does not meet the original 90% target. +The current coverage denominator also excludes some entrypoint and configuration files. +Thus, coverage should be treated as a trend signal, not as proof that tests exercise every public surface. + +`pnpm test:integration:progress` generates operational metrics. +After integration tests, CI runs `pnpm test:integration:progress:check`, so that these metrics cannot silently regress: + +- Public-command coverage. +- Device-observable workflow flag coverage. +- Public-flag classification. + +The script is the source of truth for: + +- Provider-backed integration size. +- Handler-unit size. +- Mock-heavy handler pressure. +- Public-command coverage. +- Command-family ownership. +- Device-observable workflow flag coverage. +- Provider transcript pressure. +- Low-coverage files after a coverage run. + +These flags stay in their owning unit/CLI suites: + +- Config. +- Remote transport. +- Metro preparation. +- Parser/client-only. +- Report-writing. +- Boot-fallback. + +The report lists these flags as explicit exclusions, so they are not silently missing from the denominator. +Provider pressure separates semantic Apple `simctl`/`devicectl`, macOS helper, and macOS host usage from generic Apple host-tool usage. +It also separates semantic Linux desktop/accessibility/clipboard/screenshot/input usage from generic Linux tool usage. +Thus, remote-adapter pressure is visible, and named subproviders do not count as raw shell intent. + +The progress CLI should stay a thin report and check runner over a progress model. +The model owns discovery, coverage classification, provider-pressure accounting, and check-failure derivation. +The CLI owns Markdown output and process exit behavior. +The progress script runs as a Node type-stripped TypeScript script. +When the metadata API exposes the necessary facet, the script should consume command metadata directly. +Source parsing inside the model is an implementation detail, not the desired long-term interface. +Source parsing should remain limited to facets that runtime metadata does not represent yet. +One example is the map from typed client method calls back to command names. + +Every public command should have at least one provider-backed integration scenario that runs through the request router and the request-scoped provider seams. +Unit tests remain for: + +- Parser matrices. +- Selector matching. +- Capability maps. +- Malformed inputs. +- State machines. +- Cleanup behavior. +- Provider scope routing. +- Platform error boundaries. + +Future work should be justified from live pressure in the progress script, from failing tests, or from concrete adapter needs. +A standing refactor backlog is not a justification for future work. diff --git a/docs/adr/0002-persistent-platform-helper-sessions.md b/docs/adr/0002-persistent-platform-helper-sessions.md index 4894081a3f..276fa8f198 100644 --- a/docs/adr/0002-persistent-platform-helper-sessions.md +++ b/docs/adr/0002-persistent-platform-helper-sessions.md @@ -4,118 +4,159 @@ Accepted -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Persistent protocols, -> invalidation, and explicitly safe one-shot fallback remain accepted. "Daemon-owned" means the -> daemon owns helper lifetime policy; platform modules own process-lived helper managers and helper -> mechanics, and request bindings attach to but never own or stop healthy helper generations. +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Persistent protocols, invalidation, and explicitly safe one-shot fallback remain accepted. +> In this ADR, "daemon-owned" means that the daemon owns helper lifetime policy. +> Platform modules own process-lived helper managers and helper mechanics. +> Request bindings attach to healthy helper generations but never own or stop them. -## Context +## Rules at a glance -Some platform automation backends are expensive to start but cheap to reuse. iOS already uses a -long-lived XCTest runner session with an HTTP transport. That model avoids paying `xcodebuild`, -runner boot, and XCTest readiness costs for every command, while still allowing the daemon to -invalidate the runner when the device, app, bundle, or runner process changes. +- When startup costs are high and an automation context is reusable, use persistent platform helper sessions. +- The daemon owns helper lifetime policy. Platform modules own helper-manager mechanics. +- Unless a separate cache contract has explicit invalidation, a helper must execute each command against fresh platform state. +- On Android, snapshot capture, gesture viewport resolution, and planned-touch injection share one bundled automation helper. +- On Android, after a failed session command, retry one-shot only for idempotent reads, and only after the failed session has stopped. +- On Android, return a session gesture failure directly. Do not retry it one-shot. +- For Android snapshots, if the packaged helper is unavailable or invalid, agent-device must fail closed. It must not substitute the legacy `adb uiautomator dump` snapshot engine. -Android snapshot capture initially used a one-shot instrumentation helper. Every snapshot launched -`adb shell am instrument`, connected `UiAutomation`, captured the tree, emitted XML, and exited. -Recent Android snapshot optimizations reduced XML size, idle waiting, extra file I/O, and hidden -content hint work, but a throwaway prototype still showed that process/session startup dominates -steady-state latency: +## Context -- launcher snapshot: one-shot p50 `227ms`, persistent socket p50 `5.8ms` -- React Navigation playground snapshot: one-shot p50 `265.7ms`, persistent socket p50 `16.5ms` +Some platform automation backends have high startup costs but low reuse costs. +iOS already uses a long-lived XCTest runner session with an HTTP transport. +This model avoids repeated `xcodebuild`, runner boot, and XCTest readiness costs for each command. +The daemon can still invalidate the runner when the device, app, bundle, or runner process changes. -The same pressure can appear on new platform adapters. HarmonyOS or other device backends may have -host tools, test runners, accessibility services, or bridge processes with the same shape: expensive -startup, cheap repeated commands, and a need for strict invalidation. +Android snapshot capture initially used a one-shot instrumentation helper. +Each snapshot performed these operations: -## Decision +1. Start `adb shell am instrument`. +2. Connect `UiAutomation`. +3. Capture the tree. +4. Emit XML. +5. Exit. -Use persistent platform helper sessions when a backend has high startup cost and a reusable -automation context. - -A helper session is an optimization layer governed by daemon-owned lifetime policy, not a -replacement for command correctness. Platform modules own helper-manager mechanics. A helper may -keep processes, sockets, runner state, accessibility service flags, or device forwards warm. It must -still execute each command against fresh platform state unless a separate cache contract has explicit -invalidation. - -The session pattern is: - -- start lazily on the first command that benefits from reuse -- bind the session to a device identity and helper/runner identity -- communicate through a small validated protocol with request ids and version metadata -- reuse the session while the identity and protocol remain valid -- invalidate on device disconnect, helper reinstall/version change, process exit, socket/protocol - failure, app/session identity change, or capture options that affect command semantics -- fall back to the existing one-shot path for the current command when reuse fails -- make shutdown best effort and make stale sessions disposable - -For Android snapshots, productize a persistent helper mode that keeps `UiAutomation` alive and -serves fresh snapshot requests over an `adb forward` socket. Do not add snapshot result caching as -part of that first step. The first reliable win is infrastructure reuse, not data reuse. The current -implementation keeps the existing one-shot instrumentation helper as the fallback for startup, -socket, protocol, and request failures. Both transports execute the same packaged helper contract; -agent-device must fail closed when that helper is unavailable or invalid instead of substituting -the legacy `adb uiautomator dump` snapshot engine. - -Android permits only one reliable instrumentation-owned `UiAutomation` context per device. Snapshot -capture, gesture viewport resolution, and planned-touch injection therefore share one bundled -automation helper: a live persistent helper session executes touch commands directly over its -session socket, and without one the same helper runs one-shot (amended 2026-07, issue #1275, -consistent with ADR 0013; previously touch synthesis shipped as a separate instrumentation helper, -so the daemon had to stop the persistent snapshot session before every gesture and let the next -snapshot restart it lazily). One-shot retry after a failed session command applies only to -idempotent reads such as viewport resolution, and only after the failed session has been stopped. -Gesture injection is not idempotent — events may already be partially injected — so a session -gesture failure surfaces directly instead of retrying one-shot. Helper reuse must never turn -process ownership into cross-command interference. - -For iOS, keep the XCTest runner session as the reference implementation for lifecycle and -invalidation behavior. Android does not need to copy iOS internals, but it should reuse the same -lifetime-policy ideas: per-device session management, readiness checks, structured protocol errors, -fallback/invalidation, and request-scoped observability. - -For future platforms such as HarmonyOS, prefer designing adapters around this same helper-session -contract when their native automation layer is runner-like. Avoid embedding platform-specific -startup assumptions directly in command handlers. +Recent Android snapshot optimizations reduced XML size, idle waits, extra file I/O, and work on hidden content hints. +However, a disposable prototype showed that process/session startup dominates steady-state latency: -## Alternatives Considered +- Launcher snapshot: one-shot p50 `227ms`, persistent socket p50 `5.8ms`. +- React Navigation playground snapshot: one-shot p50 `265.7ms`, persistent socket p50 `16.5ms`. -- Keep one-shot helpers only: simplest and robust, but Android measurements show it leaves an order - of magnitude of steady-state snapshot performance on the table. -- Cache snapshots in the daemon: faster for repeated reads, but unsafe after mutations, animations, - navigation, system dialogs, or app process changes unless a mutation generation contract exists. - Cache infrastructure can be added later; it should not be mixed with helper-session reuse. -- Promote an abstract cross-platform runner immediately: tempting, but premature. iOS XCTest, - Android instrumentation, macOS helper, Linux AT-SPI, and future HarmonyOS backends have different - startup and transport mechanics. Share the daemon lifecycle contract first, then extract common - code only where repetition appears. -- Replace Android instrumentation with a normal app service: potentially useful, but Android - `UiAutomation` access is instrumentation-owned. A persistent instrumentation process keeps the - required privilege model while removing repeated process startup. +New platform adapters can have the same costs. +HarmonyOS or other backends may use host tools, test runners, accessibility services, or bridge processes with this pattern: -## Consequences +- High startup costs. +- Low costs for repeated commands. +- A need for strict invalidation. -Persistent helper sessions should be measured before being productized. A prototype or benchmark -should show meaningful wall-clock improvement on a realistic app state, not just a trivial screen. +## Decision -Session managers need more lifecycle tests than one-shot helpers: startup, ready protocol, reuse, -timeout, malformed response, helper version mismatch, device disconnect, install invalidation, -shutdown, exclusive instrumentation handoff, and one-shot fallback. +When a backend has high startup costs and a reusable automation context, use persistent platform helper sessions. + +A helper session is an optimization under daemon-owned lifetime policy. +It does not replace command correctness. +Platform modules own helper-manager mechanics. +A helper may retain processes, sockets, runner state, accessibility service flags, or device forwards for reuse. +Unless a separate cache contract defines explicit invalidation, the helper must execute each command against fresh platform state. + +Use this session pattern: + +- On the first command that benefits from reuse, start the session lazily. +- Bind the session to a device identity and helper/runner identity. +- Communicate through a small validated protocol with request ids and version metadata. +- While the identity and protocol remain valid, reuse the session. +- When an invalidation condition occurs, invalidate the session. +- If reuse fails, use the existing one-shot path as the fallback for the current command. +- Make shutdown best effort. +- Make stale sessions disposable. + +These conditions invalidate a session: + +- Device disconnect. +- Helper reinstall or version change. +- Process exit. +- Socket or protocol failure. +- App or session identity change. +- Capture options that affect command semantics. + +For Android snapshots, productize a persistent helper mode that retains `UiAutomation` and serves fresh snapshot requests over an `adb forward` socket. +Do not add snapshot result caching in this first step. +The first reliable improvement is infrastructure reuse, not data reuse. +The current implementation retains the existing one-shot instrumentation helper as the fallback for startup, socket, protocol, and request failures. +Both transports execute the same packaged helper contract. +If that helper is unavailable or invalid, agent-device must fail closed. +It must not substitute the legacy `adb uiautomator dump` snapshot engine. + +Android permits only one reliable instrumentation-owned `UiAutomation` context per device. +Thus, snapshot capture, gesture viewport resolution, and planned-touch injection share one bundled automation helper (amended 2026-07, issue #1275, consistent with ADR 0013). +A live persistent helper session executes touch commands directly over its session socket. +Without a live persistent session, the same helper runs one-shot. +Before that amendment, touch synthesis shipped as a separate instrumentation helper. +Thus, the daemon had to stop the persistent snapshot session before every gesture, and the next snapshot restarted it lazily. +After a failed session command, one-shot retry applies only to idempotent reads, such as viewport resolution. +The retry runs only after the failed session has stopped. +Gesture injection is not idempotent because events may already be partially injected. +Thus, session gesture failures return directly without a one-shot retry. +Helper reuse must never let process ownership cause interference between commands. + +For iOS, keep the XCTest runner session as the reference implementation for lifecycle and invalidation behavior. +Android does not need to copy iOS internals. +However, it should reuse these lifetime-policy concepts: + +- Per-device session management. +- Readiness checks. +- Structured protocol errors. +- Fallback and invalidation. +- Request-scoped observability. + +For future platforms such as HarmonyOS, prefer this helper-session contract when the native automation layer uses a runner-like model. +Avoid platform-specific startup assumptions in command handlers. -Observability should report whether a command used a persistent session, started one, reused one, -invalidated one, or fell back to one-shot. This keeps CI and user bug reports diagnosable when a -fast path fails. +## Alternatives Considered -Persistent sessions should not make direct interactive commands unexpectedly slow. Use short -connect/request timeouts for the persistent path, then fall back to the existing one-shot timeout -budget. +- **Keep only one-shot helpers.** This option is the simplest, and it is robust. + However, Android measurements show that it loses an order of magnitude of steady-state snapshot performance. +- **Cache snapshots in the daemon.** This accelerates repeated reads. + Without a mutation generation contract, it is unsafe after mutations, animations, navigation, system dialogs, or app process changes. + Cache infrastructure can be added later. + It should remain separate from helper-session reuse. +- **Create an abstract cross-platform runner immediately.** This is premature. + iOS XCTest, Android instrumentation, macOS helper, Linux AT-SPI, and future HarmonyOS backends have different startup and transport mechanics. + Share the daemon lifecycle contract first. + Extract common code only where repetition appears. +- **Replace Android instrumentation with a normal app service.** This is potentially useful. + However, Android `UiAutomation` access belongs to instrumentation. + A persistent instrumentation process retains the required privilege model and removes repeated process startup. -The daemon remains the owner of helper lifetime policy. Platform modules own helper-manager mechanics -and may expose helper-session operations, but request bindings and command handlers must not directly -own or stop long-lived helper processes or raw host tool state. +## Consequences -This ADR does not require every backend to implement a persistent session. It defines the preferred -shape when the backend has the same startup/reuse economics that iOS and Android snapshots now -demonstrate. +Before persistent helper sessions become product features, they should be measured. +A prototype or benchmark should show meaningful wall-clock improvement on a realistic app state, not only a trivial screen. + +Session managers need more lifecycle tests than one-shot helpers. +These tests cover: + +- Startup. +- Ready protocol. +- Reuse. +- Timeout. +- Malformed response. +- Helper version mismatch. +- Device disconnect. +- Install invalidation. +- Shutdown. +- Exclusive instrumentation handoff. +- One-shot fallback. + +Observability should report whether a command used, started, reused, or invalidated a persistent session, or used a one-shot fallback. +When a fast path fails, this information keeps CI results and user bug reports diagnosable. + +Persistent sessions should not make direct interactive commands unexpectedly slow. +Use short connect/request timeouts for the persistent path. +Then use the existing one-shot timeout budget for the fallback. + +The daemon remains the owner of helper lifetime policy. +Platform modules own helper-manager mechanics and may expose helper-session operations. +Request bindings and command handlers must not directly own or stop long-lived helper processes or raw host tool state. + +This ADR does not require every backend to implement a persistent session. +It defines the preferred design when a backend has the startup and reuse costs demonstrated by iOS and Android snapshots. diff --git a/docs/adr/0003-daemon-command-registry.md b/docs/adr/0003-daemon-command-registry.md index 4d24859c03..e909deb7e8 100644 --- a/docs/adr/0003-daemon-command-registry.md +++ b/docs/adr/0003-daemon-command-registry.md @@ -4,86 +4,117 @@ Accepted +## Rules at a glance + +- Keep public command identity in `packages/command-registry/src/catalog.ts` and public input/output contracts in `src/commands/**`. +- Keep daemon route ownership and request-policy traits in the daemon command registry. +- Request modules consume named predicates from the registry. They do not recreate command string sets. +- Handler modules own execution logic only. They do not export duplicate coverage tables to prove route membership. +- The daemon registry must not define CLI grammar, Node.js client options, MCP schemas, user-facing help, or platform capability support. +- A derived daemon registry is permitted only if it preserves all four invariants in the 2026-06 update. + ## Context -Daemon request handling depends on command traits that are not part of the public command surface: -which handler route owns a command, whether tenant lease admission applies, whether session -execution should lock, whether selector validation applies, whether replay can run an action in the -current session scope, whether invalid recordings block the request, whether Android blocking-dialog -recovery applies, and how request-scoped providers resolve a device. +Daemon request behavior depends on traits outside the public command surface: + +- Which handler route owns a command. +- Whether tenant lease admission applies. +- Whether session execution should lock. +- Whether selector validation applies. +- Whether replay can run an action in the current session scope. +- Whether invalid recordings block the request. +- Whether Android blocking-dialog recovery applies. +- How request-scoped providers resolve a device. -Those traits used to be spread across `src/command-catalog.ts`, request-policy modules, and -handler-local coverage tables. That made `src/command-catalog.ts` carry daemon-only behavior next -to public command identity, and it required duplicate command sets to stay aligned by convention. +Previously, `src/command-catalog.ts`, request-policy modules, and handler-local coverage tables each held some of these traits. +Thus, `src/command-catalog.ts` contained daemon-only behavior beside public command identity. +Duplicate command sets required alignment by convention. ## Decision -Keep public command identity in `src/command-catalog.ts` and public input/output contracts in -`src/commands/**`. +Keep public command identity in `packages/command-registry/src/catalog.ts`. +Keep public input/output contracts in `src/commands/**`. + +Add `packages/daemon/src/core/daemon-command-registry.ts` as the daemon-side source of truth for command route ownership and request-policy traits. +Request modules consume predicate functions from the registry. +They do not recreate command string sets. +Handler modules own execution logic only. +They do not export duplicate coverage tables to prove route membership. + +The daemon registry is internal-only. +It must not define: -Add `packages/daemon/src/core/daemon-command-registry.ts` as the daemon-side source of truth for command route -ownership and daemon request-policy traits. Request modules consume predicate functions from the -registry instead of recreating command string sets. Handler modules own execution logic only; they do -not export duplicate coverage tables to prove route membership. +- CLI grammar. +- Node.js client options. +- MCP schemas. +- User-facing help. +- Platform capability support. -The daemon registry is internal-only. It must not define CLI grammar, Node.js client options, MCP -schemas, user-facing help, or platform capability support. Those remain owned by the command -contract, projection, help, and capability modules. +The command contract, projection, help, and capability modules retain these responsibilities. ## Alternatives Considered -- Keep daemon groups in `src/command-catalog.ts`: this keeps one command-name file, but it mixes - public command identity with daemon runtime policy and makes the catalog grow for internal-only - routing decisions. -- Keep handler-local coverage tables: this makes each handler self-describing, but creates a second - route membership source that can drift from the router and request-policy modules. -- Put route checks directly in request modules: this is locally simple, but scatters command - classification across admission, locking, provider scoping, replay, recording, and generic - dispatch. +- **Keep daemon groups in `src/command-catalog.ts`.** This retains one file for command names. + However, it mixes public command identity with daemon runtime policy. + It also increases catalog size for internal-only route decisions. +- **Keep handler-local coverage tables.** Each handler describes its routes locally. + However, this creates a second source of route membership that can disagree with the router and request-policy modules. +- **Put route checks directly in request modules.** This is locally simple. + However, command classification becomes distributed across admission, locking, provider scope, replay, recording, and generic dispatch. ## Consequences -Adding or moving a daemon-handled command requires updating the daemon command registry with its -route and request-policy traits. The registry tests pin the trait decisions, while provider-backed -integration scenarios verify important request-policy behavior through the real daemon request path. +When a daemon-handled command is added or moved, its route and request-policy traits require updates in the daemon command registry. +The registry tests pin the trait decisions. +Provider-backed integration scenarios verify important request-policy behavior through the real daemon request path. -The registry file is intentionally a dense internal contract. Its interface should stay small: -callers ask daemon-policy questions through named predicates rather than reading or mutating command -sets. +The registry file is intentionally a dense internal contract. +Its interface should remain small. +Callers ask daemon-policy questions through named predicates. +They do not read or mutate command sets. -`AGENTS.md` should contain only the operating rule and relevant file pointers for agents. This ADR -owns the rationale so future changes do not need to infer it from agent instructions. +`AGENTS.md` should contain only the operating rule and relevant file pointers for agents. +This ADR owns the rationale. +Thus, future changes do not require readers to infer the rationale from agent instructions. ## Update (2026-06): single-declaration / derivation model -A later proposal (the `CommandDescriptor` direction, now [ADR 0008](0008-command-descriptor-registry.md)) unifies a command's -declarations so the public catalog, capability matrix, CLI/MCP projections, batch allowlist, and this -daemon registry are *derived* from one registration site, to remove the cross-table drift that several -of these surfaces are kept aligned against by convention. - -**This ADR's decision stands.** Its boundary is about *ownership* and the *predicate interface*, not -about the physical file a trait is typed in. "Separate source of truth" means separately owned and -exposed through named predicates — a property that survives a projected/derived backing table. A -derived daemon registry is therefore permitted **only if** it preserves all of the following invariants: - -1. **Daemon-owned declaration.** Route and request-policy traits are declared in a daemon-owned facet - (under `packages/daemon/src/`) and *composed* into the registration — never inlined as fields on the public - command contract in `src/commands/**` or `src/command-catalog.ts`. Co-locating a registration *call* - is fine; co-locating *ownership* of daemon policy in the public surface is the contamination this ADR - rejected (see Alternatives, "Keep daemon groups in `src/command-catalog.ts`"). -2. **Predicate interface unchanged.** Consumers keep asking daemon-policy questions through the named - predicates (`getDaemonCommandRoute`, `isLeaseAdmissionExempt`, `shouldLockSessionExecution`, …). The - daemon registry remains their sole exposer; derivation changes how the backing table is *built*, not - how it is *read*. -3. **No leakage into public projections.** The catalog/CLI/MCP/help/capability projections must be - type-prevented from reading daemon-only traits, and the daemon registry must still not define CLI - grammar, Node.js options, MCP schemas, user-facing help, or capability support. -4. **One declaration per concern, enforced by types.** The single registration site must make a missing - or duplicated daemon trait a *compile error* — replacing today's "aligned by convention". This is the - structural improvement that justifies derivation over a separately hand-authored table. - -A single flat public descriptor whose daemon fields leak into public views is **not** permitted — that is -the "collapse daemon policy into a public command registry" failure this ADR exists to prevent. Compose -facets owned by their domains; derive the registry from them. Until that derivation lands and is pinned -by the registry tests, the hand-authored `packages/daemon/src/core/daemon-command-registry.ts` remains the source of -truth and the operating rule in `AGENTS.md` is unchanged. +A later proposal, the `CommandDescriptor` direction in [ADR 0008](0008-command-descriptor-registry.md), unifies a command's declarations at one registration site. +It derives these surfaces from that site: + +- Public catalog. +- Capability matrix. +- CLI/MCP projections. +- Batch allowlist. +- Daemon registry. + +Its purpose is to remove the drift between these tables. +Convention keeps several of these surfaces aligned against that drift. + +**This ADR's decision remains accepted.** +Its boundary concerns ownership and the predicate interface, not the physical file that contains a trait. +“Separate source of truth” means separate ownership and exposure through named predicates. +That property remains true for a projected or derived backing table. +A derived daemon registry is therefore permitted **only if** it preserves all these invariants: + +1. **Daemon-owned declaration.** A daemon-owned facet under `packages/daemon/src/` declares route and request-policy traits. + The registration composes that facet. + The public command contract in `src/commands/**` or `packages/command-registry/src/catalog.ts` must never contain these traits as inline fields. + A registration call may be co-located with public declarations. + Ownership of daemon policy must remain separate from the public surface. + See Alternatives, “Keep daemon groups in `src/command-catalog.ts`”. +2. **Predicate interface unchanged.** Consumers continue to use named predicates such as `getDaemonCommandRoute`, `isLeaseAdmissionExempt`, and `shouldLockSessionExecution`. + Only the daemon registry exposes these predicates. + Derivation changes table construction, not table access. +3. **No leakage into public projections.** Types must prevent the catalog/CLI/MCP/help/capability projections from reading daemon-only traits. + The daemon registry must still not define CLI grammar, Node.js options, MCP schemas, user-facing help, or capability support. +4. **One declaration per concern, enforced by types.** At the single registration site, a missing or duplicate daemon trait must cause a compile error. + This replaces alignment by convention. + This structural improvement justifies derivation instead of a separate hand-authored table. + +A single flat public descriptor that exposes daemon fields in public views is **not** permitted. +This would combine daemon policy with a public command registry, which this ADR rejects. +Compose facets owned by their domains. +Derive the registry from those facets. +Until that derivation is implemented and the registry tests pin it, the hand-authored `packages/daemon/src/core/daemon-command-registry.ts` remains the source of truth. +Until then, the operating rule in `AGENTS.md` does not change. diff --git a/docs/adr/0004-ios-snapshot-backend-strategy.md b/docs/adr/0004-ios-snapshot-backend-strategy.md index 9657ecd3d1..723a07e8b9 100644 --- a/docs/adr/0004-ios-snapshot-backend-strategy.md +++ b/docs/adr/0004-ios-snapshot-backend-strategy.md @@ -2,550 +2,651 @@ ## Status -Accepted. Amended after local iOS Simulator acquisition moved to the host AX bridge while the -public surface remained two modes: regular interactive snapshots and raw diagnostic snapshots. +Accepted. +This ADR was amended after local iOS Simulator acquisition moved to the host AX bridge. +The public surface remains two modes: regular interactive snapshots and raw diagnostic snapshots. +The host AX bridge is an acquisition route, not a third public mode. + +Completed process history is in git history. The Apple platform runtime owns acquisition routing and its generation-scoped XCTest fallback. -Host-side iOS validation, semantic presentation, and publication are owned by -`@agent-device/capture-kit`; structured snapshot quality verdicts and fallback warnings make -degraded or recovered output observable end to end. +`@agent-device/capture-kit` owns host-side iOS validation, semantic presentation, and publication. +Structured snapshot quality verdicts and fallback warnings make degraded or recovered output observable end to end. + +## Rules at a glance + +- Route eligible local iOS Simulator snapshots through the host AX bridge. Present them once through the shared TypeScript engine. +- If bridge acquisition or presentation fails, use one typed XCTest fallback. +- After fallback, disable the bridge for that app generation, except for capture-scoped failures. A new app generation enables it again. +- A refusal that provides evidence about the screen, not the app, has capture scope. +- Keep the regular visible strategy and the raw diagnostic strategy explicit as the two public snapshot strategies. +- The regular visible strategy must not use an arbitrary node-count cutoff. +- If the transport cannot carry a raw response, fail explicitly instead of silently truncating the tree at a hard node count. +- New consumers must use the host-side snapshot policy facet in `@agent-device/capture-kit` instead of adding another daemon presentation path. +- Acquisition backends serialize facts. Presentation applies visibility, eligibility, scope, and the regular depth cut. +- Observe and operate a registered system-surface host in place. Never activate it. ## Context -Agent Device exposes iOS UI state through host AX acquisition on local Simulators and the long-lived -XCTest runner everywhere else. The snapshot surface has two durable needs: +Agent Device exposes iOS UI state through host AX acquisition on local Simulators and the long-lived XCTest runner everywhere else. +The snapshot surface has two durable needs: -- agent-facing regular context, where the important contract is the effective user-visible UI, - fixed controls such as tab bars, and scroll-hidden hints for content outside visible scroll - containers; -- rich diagnostics and selector disambiguation, where a raw recursive XCTest snapshot is useful - because it preserves hierarchy, static text, wrappers, scroll containers, and ancestry. +- Agent-facing regular context. Here the important contract is the effective user-visible UI, fixed controls such as tab bars, and scroll-hidden hints for content outside visible scroll containers. +- Rich diagnostics and selector disambiguation. Here a raw recursive XCTest snapshot is useful because it preserves hierarchy, static text, wrappers, scroll containers, and ancestry. -These needs should not share one capture strategy blindly. Recursive `XCUIElement.snapshot()` is -rich, but some real app trees can make XCTest fail with `kAXErrorIllegalArgument` or main-thread -timeouts while the same app remains visually usable. Bluesky is the current known example: -lower-level accessibility services can describe simulator screens even when XCTest recursive -snapshots and typed `XCUIElementQuery` enumeration degrade to no useful child nodes. Physical iOS -devices can show the same XCTest accessibility-channel timeout shape even when no lower-level -semantic backend is available. +These needs should not share one capture strategy without distinction. +Recursive `XCUIElement.snapshot()` is rich. +However, some real app trees can make XCTest fail with `kAXErrorIllegalArgument` or main-thread timeouts while the same app remains visually usable. +Bluesky is the current known example. +Lower-level accessibility services can describe simulator screens even when recursive XCTest snapshots and typed `XCUIElementQuery` enumeration degrade to no useful child nodes. +Physical iOS devices can show the same XCTest accessibility-channel timeout shape even when no lower-level semantic backend is available. -This is different from presentation filtering. The daemon's snapshot presentation can hide noisy -or inaccessible nodes, but it cannot recover nodes that XCTest never returns. More filters, -Maestro-specific heuristics, or retries in the daemon would only make this failure slower and less -predictable. +This failure is different from presentation filtering. +The daemon's snapshot presentation can hide noisy or inaccessible nodes. +It cannot recover nodes that XCTest never returns. +More filters, Maestro-specific heuristics, or retries in the daemon would only make this failure slower and less predictable. ## Decision -Keep XCTest as the iOS automation runner. Route eligible local iOS Simulator snapshots through the -host AX bridge, present them once through the shared TypeScript engine, and use one typed XCTest -fallback when bridge acquisition or presentation fails. Disable the bridge for that app generation -after fallback; a new app generation re-enables it. Physical devices, providers, custom-action -captures, and interactions remain on their existing owners. - -A WebKit page — Safari's, or a `WKWebView`'s — lives in a WebContent process and reaches UIKit's -tree as an `AXRemoteElement` under the web view, with its children in that other process. The -bridge reads one process, so it delivers the element as a leaf. The source refuses a tree in which -such a leaf sits under a `WebView`-typed ancestor and reaches the viewport (`remote-content-boundary`) -instead of publishing a screen without its page: refs issued from it would target the host views -around the page rather than the page. A leaf whose frame is zero-area or off screen hosts nothing -the capture can miss and is published; one that reports no frame is refused, because nothing proves -it empty. A scene-hosted remote view controller — a share or action extension presented over the -host app — reaches the tree the same way: an `AXRemoteElement` leaf under a `_UISceneHostingView`, -with the extension's controls in its own process. The source refuses that leaf by the same rule. -Remote elements under any other host are not classified — no capture has shown one — and a host -truncated away by the node or depth cap stays disclosed as truncation. XCTest resolves remote -elements, so the fallback serves the page or the extension (#2484). - -The refusal opens the generation circuit, as any failure that says something about the app itself -does, so a hybrid app that showed one web screen takes XCTest for its remaining native screens until -it relaunches — the 0.20.x path for every screen. Re-asking the bridge per capture would instead -charge a refused bridge round trip to every `wait` poll on the web screen; the circuit keeps that cost -to one capture per generation. A refusal that says something about the screen rather than the app is -scoped per capture instead; the coordinate-space amendment below decides that case. +Keep XCTest as the iOS automation runner. +Route eligible local iOS Simulator snapshots through the host AX bridge. +Present them once through the shared TypeScript engine. +If bridge acquisition or presentation fails, use one typed XCTest fallback. +After fallback, disable the bridge for that app generation, subject to the capture-scoped exception below. +A new app generation enables it again. +Physical devices, providers, custom-action captures, and interactions remain on their existing owners. + +A WebKit page in Safari or in a `WKWebView` lives in a WebContent process. +It reaches UIKit's tree as an `AXRemoteElement` under the web view. +Its children are in that other process. +The bridge reads one process, so it delivers the element as a leaf. +If such a leaf has a `WebView`-typed ancestor and reaches the viewport, the source refuses the tree (`remote-content-boundary`). +The source refuses the tree instead of publishing a screen without its page. +If the source published that tree, refs issued from it would target the host views around the page instead of the page. +If the leaf frame is zero-area or off screen, the leaf hosts nothing that the capture can miss, so the source publishes it. +If the leaf reports no frame, the source refuses it because nothing proves it empty. + +A scene-hosted remote view controller (a share or action extension presented over the host app) reaches the tree in the same way. +It is an `AXRemoteElement` leaf under a `_UISceneHostingView`. +The extension's controls are in its own process. +The source refuses that leaf by the same rule. +Remote elements under any other host are not classified. +No capture has shown one. +If the node cap or the depth cap removes the host, the capture still discloses it as truncation. +XCTest resolves remote elements, so the fallback serves the page or the extension (#2484). + +The refusal opens the generation circuit, as any failure that provides evidence about the app itself does. +Thus, after a hybrid app shows one web screen, its remaining native screens use XCTest until the app relaunches. +In 0.20.x, every screen used this XCTest path. +A bridge request on each capture would instead add a refused bridge round trip to every `wait` poll on the web screen. +The circuit limits that cost to one capture per generation. +A refusal that provides evidence about the screen instead of the app has capture scope. +The coordinate-space amendment below decides that case. Keep the two public snapshot strategies explicit: -- **Regular visible strategy**: use recursive XCTest snapshots, emit the effective user-visible - tree plus visible ancestors and scroll-hidden hints, and fall back through the capture plan when - XCTest returns sparse output. A node inside a scroll container is user-visible only when it - intersects both the app viewport and the nearest visible scroll container. Offscreen descendants - should be visited to set `hiddenContentAbove` / `hiddenContentBelow`, not emitted as normal - visible nodes. This strategy must not use an arbitrary node-count cutoff: fixed controls that are - later in traversal order, such as bottom tab bars after long lists, are part of the visible UI - contract. -- **Raw diagnostic strategy**: use recursive XCTest snapshots for raw snapshots, diagnostics, and - cases that need hierarchy. Raw output is allowed to be noisy and large; if the transport cannot - carry the response, fail explicitly instead of silently truncating the tree at a hard node count. - If XCTest reports a real AX serialization failure, preserve that error instead of pretending the - UI is empty. +- **Regular visible strategy**: use recursive XCTest snapshots. + Emit the effective user-visible tree, visible ancestors, and scroll-hidden hints. + If XCTest returns sparse output, fall back through the capture plan. + A node inside a scroll container is user-visible only if it intersects both the app viewport and the nearest visible scroll container. + Offscreen descendants should be visited to set `hiddenContentAbove` / `hiddenContentBelow`. + They should not be emitted as normal visible nodes. + This strategy must not use an arbitrary node-count cutoff. + Fixed controls that are later in traversal order, such as bottom tab bars after long lists, are part of the visible UI contract. +- **Raw diagnostic strategy**: use recursive XCTest snapshots for raw snapshots, diagnostics, and cases that need hierarchy. + Raw output is allowed to be noisy and large. + If the transport cannot carry the response, fail explicitly instead of silently truncating the tree at a hard node count. + If XCTest reports a real AX serialization failure, preserve that error instead of reporting an empty UI. - **Host AX strategy**: acquire local Simulator trees as raw facts through the bounded host bridge. - Every result crosses the same presentation boundary before publication. XCTest fallback carries - explicit source residue, and comparisons require matching producer, intent, app generation, - presentation key, and residue. Physical devices should use an equivalent non-XCTest semantic - backend only if Apple exposes a supported channel. + Before publication, every result crosses the same presentation boundary. + XCTest fallback carries explicit source residue. + Comparisons require matching producer, intent, app generation, presentation key, and residue. + Physical devices should use an equivalent non-XCTest semantic backend only if Apple exposes a supported channel. -The daemon should make degraded output observable. If an iOS interactive snapshot contains only the -application root or another sparse shape, surface a structured quality verdict and warning so -agents know the snapshot is degraded output rather than proof that the screen has no controls. +The daemon should make degraded output observable. +If an iOS interactive snapshot contains only the application root or another sparse shape, return a structured quality verdict and warning. +Thus, agents know that the snapshot is degraded output, not proof that the screen has no controls. ## Host-side ownership boundary -The shared TypeScript side has one snapshot-presentation facet. The neutral acquisition-to-presented -carrier and clip-fold geometry contract live in `@agent-device/contracts/snapshot-presentation`, while -`@agent-device/capture-kit` owns host-side iOS planning, folding, projection, eligibility, semantic -compaction, validation, and publication. Platform-specific presentation adapters retain only the -policy mechanics that cannot yet cross their runtime boundary. Daemon assembly owns only the ordering -of capture, compaction, occlusion, and ref publication. It does not own the presentation vocabulary -or a second geometry carrier. - -Android acquisition remains in its platform module and adapts its raw hierarchy to the shared -carrier. Swift keeps its runner-side `SnapshotPresentation` implementation because it consumes the -capture-plan tier before the process boundary. The iOS engine fixture is the shared proof between -those runtimes; it does not imply that Swift and TypeScript share an implementation. -The macOS XCTest runner is the desktop-surface exception: its already-presented nodes bypass the iOS -presentation engine and continue through neutral snapshot assembly. - -The same split now holds for the three remaining Wave 4 policies tracked by #1983, so the host-side -facet in `@agent-device/capture-kit` owns snapshot policy generally rather than presentation alone: - -- **Freshness recovery.** The freshness window, the Android staleness classification and its - thresholds, and the retry loop live in `packages/capture-kit/src/snapshot/snapshot-freshness/`. - The loop is parameterized by a classifier and a retry schedule, so "how long may a backend lag - behind a real transition" is a policy input rather than a constant the loop owns. The schedule is - stated as a duration budget; the loop derives the deadline from the window's `markedAt` itself, so - the budget is always spent from the action and a caller has no absolute instant it could get - wrong. - `packages/daemon/src/core/session-snapshot-freshness.ts` keeps only what needs a session: reading and retiring - the window on store-owned `SessionState`, and choosing the comparison baseline from snapshot - lineage. It remains the declared R7 owner of `androidSnapshotFreshness`. -- **Timeout evidence.** Whether a capture failed because the hierarchy never arrived is decided - once, at the deepest boundary that has the evidence, from machine-defined values only: - `snapshot-capture-failure-reason.ts` maps the helper's structured `errorType` field - (`java.util.concurrent.TimeoutException`, by exact equality) and the SIGKILL exit code 137 to - the typed reason `accessibility-timeout` (`ANDROID_CAPTURE_FAILURE_REASONS` in - `@agent-device/contracts/android-snapshot-quality`). The helper-result, session-protocol, and - killed-instrumentation error constructors attach it; every layer above rewraps it rather than - reclassifying. No message shape is consulted anywhere on that path, so rewording helper or - wrapper prose cannot move the reason, and prose that merely reads like a timeout does not become - one — both directions are asserted end to end against the real producer. - `packages/capture-kit/src/snapshot/snapshot-timeout-policy.ts` reads the reason; the - human-facing hint is derived from it rather than decided alongside it. - - The published `details.androidSnapshotTimeoutScreenshot` payload is vocabulary in - `@agent-device/contracts/snapshot-timeout-evidence`, a union whose arms encode which claims can - coexist. The annotated arm carries a non-empty ref tuple, so "annotated with zero refs" is not a - state a caller can build, and no arm stores a ref count: a count beside the refs is a second - source of truth the type system cannot hold in step, so it is derived from the refs instead. - The daemon keeps the ordering that genuinely needs it: resolving a bound screenshot runtime, - writing the artifact, annotating it from the stored observation, and emitting the diagnostics. -- **Screenshot-overlay policy.** Which Android nodes earn an overlay ref, and what rectangle an - overlay for one of them covers, live in `packages/capture-kit/src/screenshot-overlay*.ts`. The - daemon keeps approved artifact and ref assembly only: ranking, projection to screenshot pixels, - drawing, and PNG IO. - -`scripts/layering/snapshot-presentation-boundary.test.ts` enforces the direction, but only across -the roots `snapshot-policy` declares in `scripts/layering/architecture-ownership.ts`: nothing in -that snapshot tree may import `packages/daemon/src/`, while the overlay modules beside the tree sit outside -those roots and are outside that gate. It carries a positive control, because a filter that stopped -matching would look identical to a boundary being obeyed. - -The residual call sites #1983 also named are audited and deliberately left in place. -`packages/daemon/src/core/direct-ios-selector.ts` carries no presentation policy: `isLocalIosRunnerSession` and -`readSimpleIosSelectorTarget` are session routing (device family, provider ownership, the -stabilization window), while `deriveDirectIosNodeSelector` and `isDirectIosSelectorFallbackError` -are selector derivation and ADR 0011 delegation-on-error. The latter two are pure and -daemon-independent, but their owner would be the selector pipeline governed by R19, not this -facet; moving them under ADR 0004 would widen it to a boundary it does not decide. The -observation and interaction consumers — `selector-capture-runtime.ts`, -`deferred-interaction-outcome.ts`, `snapshot-capture.ts` and -`interaction-touch-android-freshness.ts` — now reach freshness only through the facet or its -session binding. - -New consumers must use the facet rather than add another daemon presentation path. - -The acquisition/presentation boundary has two explicit vocabularies. An acquired input is raw node -evidence accompanied by its capture hint, viewport, lineage, and residue; the host engine folds and -projects that evidence. A presented input is the runner's primary payload plus validation facts; the -host engine validates it and performs semantic compaction once. Regular eligibility decides which -nodes belong in the regular presentation, while publication adds refs and emits only the primary -payload. An optional unscoped quality payload is validated for classification evidence and is never -published. - -Semantic compaction may move an identifier; it may not un-make one. A structural `Other` wrapper -carrying an identifier and nothing else is suppressed in favour of its content, which is a -delegation: the identifier goes on living in whatever the wrapper stood for. A wrapper with no -content has nothing to delegate to, so suppressing it deletes the identifier from every canonical -view while `is`, `get`, and `click` still resolve it from the same capture. That deletion needs the -node's own declared `hittable: false`, because it is the only verdict in the capture that says the -wrapper is inert; a producer that reports no hittability for any node declares nothing, and an -absent fact is not a negative answer (#2638). +The shared TypeScript side has one snapshot-presentation facet. +The neutral acquisition-to-presented carrier and the clip-fold geometry contract live in `@agent-device/contracts/snapshot-presentation`. +`@agent-device/capture-kit` owns these host-side iOS responsibilities: + +- Planning. +- Folding. +- Projection. +- Eligibility. +- Semantic compaction. +- Validation. +- Publication. + +Platform-specific presentation adapters retain only the policy mechanics that cannot yet cross their runtime boundary. +Daemon assembly owns only the order of capture, compaction, occlusion, and ref publication. +It does not own the presentation vocabulary or a second geometry carrier. + +Android acquisition remains in its platform module and adapts its raw hierarchy to the shared carrier. +Swift keeps its runner-side `SnapshotPresentation` implementation because it consumes the capture-plan tier before the process boundary. +The iOS engine fixture is the shared proof between those runtimes. +It does not imply that Swift and TypeScript share an implementation. +The macOS XCTest runner is the desktop-surface exception. +Its already-presented nodes bypass the iOS presentation engine and continue through neutral snapshot assembly. + +The same split holds for the three remaining Wave 4 policies tracked by #1983. +Thus, the host-side facet in `@agent-device/capture-kit` owns snapshot policy generally, not only presentation: + +- **Freshness recovery.** The freshness window, the Android staleness classification and its thresholds, and the retry loop live in `packages/capture-kit/src/snapshot/snapshot-freshness/`. + A classifier and a retry schedule parameterize the loop. + Thus, "how long may a backend lag behind a real transition" is a policy input, not a constant that the loop owns. + The schedule is stated as a duration budget. + The loop itself derives the deadline from the window's `markedAt`. + Thus, the budget is always spent from the action, and a caller has no absolute instant that it could get wrong. + `packages/daemon/src/core/session-snapshot-freshness.ts` keeps only the operations that need a session. + It reads and retires the window on store-owned `SessionState`, and it chooses the comparison baseline from snapshot lineage. + It remains the declared R7 owner of `androidSnapshotFreshness`. +- **Timeout evidence.** The deepest boundary that has the evidence decides once whether a capture failed because the hierarchy never arrived. + It decides from machine-defined values only. + `snapshot-capture-failure-reason.ts` maps the helper's structured `errorType` field and the SIGKILL exit code 137 to the typed reason `accessibility-timeout`. + The `errorType` match is an exact equality with `java.util.concurrent.TimeoutException`. + `ANDROID_CAPTURE_FAILURE_REASONS` in `@agent-device/contracts/android-snapshot-quality` defines the typed reason. + The helper-result, session-protocol, and killed-instrumentation error constructors attach it. + Every layer above rewraps it instead of reclassifying it. + No code on that path examines a message shape. + Thus, a change to helper or wrapper prose cannot move the reason. + Prose that only resembles a timeout does not become one. + End-to-end assertions against the real producer verify both directions. + `packages/capture-kit/src/snapshot/snapshot-timeout-policy.ts` reads the reason. + The human-facing hint is derived from the reason, not decided beside it. + + The published `details.androidSnapshotTimeoutScreenshot` payload is vocabulary in `@agent-device/contracts/snapshot-timeout-evidence`. + It is a union whose arms encode which claims can coexist. + The annotated arm carries a non-empty ref tuple, so "annotated with zero refs" is not a state that a caller can build. + No arm stores a ref count. + A count beside the refs is a second source of truth that the type system cannot keep consistent. + Thus, the count is derived from the refs instead. + The daemon keeps the ordering that genuinely needs it: + + 1. Resolve a bound screenshot runtime. + 2. Write the artifact. + 3. Annotate it from the stored observation. + 4. Emit the diagnostics. + +- **Screenshot-overlay policy.** `packages/capture-kit/src/screenshot-overlay*.ts` decides which Android nodes earn an overlay ref and which rectangle an overlay for one of them covers. + The daemon keeps only approved artifact and ref assembly: ranking, projection to screenshot pixels, drawing, and PNG IO. + +`scripts/layering/snapshot-presentation-boundary.test.ts` enforces the direction, but only across the roots that `snapshot-policy` declares in `scripts/layering/architecture-ownership.ts`. +Nothing in that snapshot tree may import `packages/daemon/src/`. +The overlay modules beside the tree are outside those roots and outside that gate. +The test carries a positive control, because a filter that stopped matching would look identical to an obeyed boundary. + +The residual call sites that #1983 also named are audited and intentionally left in place. +`packages/daemon/src/core/direct-ios-selector.ts` carries no presentation policy. +`isLocalIosRunnerSession` and `readSimpleIosSelectorTarget` are session routing (device family, provider ownership, the stabilization window). +`deriveDirectIosNodeSelector` and `isDirectIosSelectorFallbackError` are selector derivation and ADR 0011 delegation-on-error. +The latter two are pure and daemon-independent. +However, their owner would be the selector pipeline that R19 governs, not this facet. +A move under ADR 0004 would widen this ADR to a boundary that it does not decide. +These observation and interaction consumers now reach freshness only through the facet or its session binding: + +- `selector-capture-runtime.ts`. +- `deferred-interaction-outcome.ts`. +- `snapshot-capture.ts`. +- `interaction-touch-android-freshness.ts`. + +New consumers must use the facet instead of adding another daemon presentation path. + +The acquisition/presentation boundary has two explicit vocabularies. +An acquired input is raw node evidence with its capture hint, viewport, lineage, and residue. +The host engine folds and projects that evidence. +A presented input is the runner's primary payload plus validation facts. +The host engine validates it and performs semantic compaction once. +Regular eligibility decides which nodes belong in the regular presentation. +Publication adds refs and emits only the primary payload. +An optional unscoped quality payload is validated for classification evidence and is never published. + +Semantic compaction may move an identifier, but it may not delete one. +A structural `Other` wrapper that carries an identifier and nothing else is suppressed in favor of its content. +This suppression is a delegation: the identifier continues to exist in whatever the wrapper represented. +If the wrapper has no content, it has nothing to delegate to. +Thus, its suppression deletes the identifier from every canonical view while `is`, `get`, and `click` still resolve it from the same capture. +That deletion needs the node's own declared `hittable: false`. +This is the only verdict in the capture that says the wrapper is inert. +A producer that reports no hittability for any node declares nothing. +An absent fact is not a negative answer (#2638). ## Regression Notes -PR #639 made XCTest AX serialization failures explicit instead of swallowing them as empty -snapshots. That was the correct diagnostic change, but it exposed apps whose accessibility trees -XCTest cannot serialize. - -Later work moved recovery into the regular visible capture plan so healthy apps keep the fast -recursive tree path while degraded app classes can still return bounded, honest output when -fallback tiers are the only available source of visible controls. - -Issue #1105 showed a second failure shape on the same app class: instead of failing fast with -`kAXErrorIllegalArgument`, the recursive tree capture can grind for many seconds on -heavy/animating screens before failing, pushing the chained plan past the runner's main-thread -watchdog and burying the main queue under retries. The plan now carries its umbrella deadline -into the query-sweep and private-AX tiers (later ladder rungs stop when the budget is spent), -and a slow, timed-out, or watchdog-abandoned XCTest-backed capture penalizes the XCTest -accessibility channel for that bundle for a bounded window. Subsequent regular plans derive the -next step from backend traits (`effectiveSnapshotCapturePlan`): when a runnable non-XCTest backend -exists, they defer to that independent tier; when it does not, as on physical iOS devices today, -they run a short XCTest probe instead of the full tree slice so healthy screens can recover without -repeating the hostile-screen grind. The raw diagnostic plan is exempt — it keeps tree-first error -propagation. - -A third shape followed on the same app class once the plan recovered reliably. The query-sweep -tier's 19 `allElementsBoundByIndex` reads each fail with `kAXErrorIllegalArgument`, and XCTest -records every one as a test failure worded `Failed to resolve query: ...`. Any recorded failure the -runner does not mute ends `testCommand` as soon as the main-thread block that recorded it returns, -whatever `continueAfterFailure` says, so the runner died after (or during) every hostile snapshot -and the per-bundle penalty and depth memory died with it. The runner now mutes AX-server rejections -in both XCTest fetch wordings, and every bounded main-thread dispatch that outlives its slice counts -as occupying the main thread (the tree XPC and the system-modal probe previously kept a second count -of their own), so a viewport read that grinds makes the plan skip the sweep instead of queueing it. +PR #639 made XCTest AX serialization failures explicit instead of returning them as empty snapshots. +That diagnostic change was correct, but it exposed apps whose accessibility trees XCTest cannot serialize. + +Later work moved recovery into the regular visible capture plan. +Thus, healthy apps keep the fast recursive tree path. +When fallback tiers are the only available source of visible controls, degraded app classes can still return bounded, honest output. + +Issue #1105 showed a second failure shape on the same app class. +On heavy or animating screens, the recursive tree capture can work for many seconds before it fails, instead of failing fast with `kAXErrorIllegalArgument`. +This pushes the chained plan past the runner's main-thread watchdog and fills the main queue with retries. +The plan now carries its umbrella deadline into the query-sweep and private-AX tiers. +When the budget is spent, later ladder rungs stop. +A slow, timed-out, or watchdog-abandoned XCTest-backed capture penalizes the XCTest accessibility channel for that bundle for a bounded window. +Subsequent regular plans derive the next step from backend traits (`effectiveSnapshotCapturePlan`). +If a runnable non-XCTest backend exists, they defer to that independent tier. +If it does not exist, as on physical iOS devices today, they run a short XCTest probe instead of the full tree slice. +Thus, healthy screens can recover without another long capture of the hostile screen. +The raw diagnostic plan is exempt and keeps tree-first error propagation. + +A third shape followed on the same app class after the plan recovered reliably. +On that app class, each of the query-sweep tier's 19 `allElementsBoundByIndex` reads fails with `kAXErrorIllegalArgument`. +XCTest records every one as a test failure worded `Failed to resolve query: ...`. +Any recorded failure that the runner does not mute ends `testCommand` as soon as the main-thread block that recorded it returns, whatever `continueAfterFailure` says. +Thus, the runner exited after (or during) every hostile snapshot, and the per-bundle penalty and depth memory were lost with it. +The runner now mutes AX-server rejections in both XCTest fetch wordings. +Every bounded main-thread dispatch that outlives its slice counts as occupying the main thread. +Thus, if a viewport read takes too long, the plan skips the sweep instead of queueing it. ## Recovery conformance and depth hints -The host AX bridge and the XCTest runner's private AX bridge recover rejected deep requests with -different native representations, ladders, and completeness evidence, and they stay separate -implementations. `contracts/fixtures/ios-ax-recovery-conformance.json` is their shared, executable -recovery contract: each producer replays every case through its own adapter, and the fixture -records per-producer expectations plus the intentional differences, so a change to either recovery -path is measured against the same synthetic native world. Common executable policy is extracted only -where the fixture proves equivalence; a shared engine is not a goal. - -The host source additionally keeps a bounded accepted-depth hint per resolved target generation -and producer. It changes only the native levels the first request asks for, is learned only from a -finished recovery that observed a rejection, expires by hinted-capture count so ordinary screens -probe back to the full depth, and is never shared across apps, generations, or producers. The -route's generation circuit remains the only lifecycle owner. +The host AX bridge and the XCTest runner's private AX bridge recover rejected deep requests with different native representations, ladders, and completeness evidence. +They stay separate implementations. +`contracts/fixtures/ios-ax-recovery-conformance.json` is their shared, executable recovery contract. +Each producer replays every case through its own adapter. +The fixture records per-producer expectations and the intentional differences. +Thus, a change to either recovery path is measured against the same synthetic native world. +Common executable policy is extracted only where the fixture proves equivalence. +A shared engine is not a goal. + +The host source also keeps a bounded accepted-depth hint per resolved target generation and producer. +The hint changes only the native levels that the first request asks for. +It is learned only from a finished recovery that observed a rejection. +It expires by hinted-capture count, so ordinary screens probe back to the full depth. +It is never shared across apps, generations, or producers. +The route's generation circuit remains the only lifecycle owner. ## Consequences -Regular snapshots remain the right tool for agents and Maestro compatibility because they describe -what a user can currently perceive and interact with. Raw snapshots remain the right tool when -hierarchy matters. Both may still fail loudly on XCTest-broken trees; that failure is useful -because retrying the same recursive capture is unlikely to reveal a different tree. - -A future AX-service backend is the correct place to regain Bluesky-class semantic coverage. It -should be added as a platform backend with its own lifecycle, protocol, normalization, timing -metrics, and fallback rules, not as another special case inside the XCTest runner. - -The acquire/present migration begins with a behavior-preserving typed seam: acquisition backends -construct `RawAXNode`, `SnapshotPresentation` alone constructs `PresentedNode`, and response payloads -accept only presented nodes. Its second behavior-preserving step makes every capture-plan backend -return `SnapshotAcquisition` and routes the exhaustive backend switch through one -`SnapshotPresentation.present` call. `PresentationOptions` is the stable request-policy input to -that boundary. Until the remaining migration steps move interpretation into the boundary, acquisition -still reads those options and raw nodes intentionally carry the derived fields produced by the -existing backends. - -The first semantic migration layer makes regular eligibility backend-neutral inside -`SnapshotPresentation`: the top-level viewport carrier survives, and every other node needs an -interactive accessibility type or a non-empty label, identifier, or value. Hittability no longer -admits an otherwise ineligible node. Raw membership remains unchanged. This is runner eligibility, -not daemon publication membership; backend-blind daemon compaction retains ownership of its declared -noise suppressions. When eligibility removes a structural wrapper, presentation reparents its -surviving descendants to the nearest surviving ancestor and normalizes their indexes and depths. - -The second semantic layer makes scope a presentation specification rather than an acquisition or -daemon-compaction policy. A trimmed non-empty scope selects the first presentation-preorder match -whose subtree contributes to the requested projection; matching inspects label, identifier, and -value case-insensitively. The selected subtree is re-rooted, depth is applied relative to that root, -and no match publishes an empty healthy projection. Swift and TypeScript implementations are pinned -by `contracts/fixtures/snapshot-scope-policy.json`. -Scoped iOS acquisition stays broad (including when depth is requested) until an adapter can prove a -narrowing hint complete. The daemon never reapplies scope after the wire; Android selects its root -inside its TypeScript presentation and desktop surface runtimes retain their platform projection. +Regular snapshots remain the right tool for agents and Maestro compatibility because they describe what a user can currently perceive and interact with. +Raw snapshots remain the right tool when hierarchy matters. +Both may still fail explicitly on XCTest-broken trees. +That failure is useful because another identical recursive capture is unlikely to reveal a different tree. + +A future AX-service backend is the correct place to regain Bluesky-class semantic coverage. +It should be added as a platform backend with its own lifecycle, protocol, normalization, timing metrics, and fallback rules. +It should not be added as another special case inside the XCTest runner. + +The acquire/present migration begins with a typed seam that preserves behavior: + +- Acquisition backends construct `RawAXNode`. +- `SnapshotPresentation` alone constructs `PresentedNode`. +- Response payloads accept only presented nodes. + +Its second step also preserves behavior. +It makes every capture-plan backend return `SnapshotAcquisition`. +It routes the exhaustive backend switch through one `SnapshotPresentation.present` call. +`PresentationOptions` is the stable request-policy input to that boundary. + +The first semantic migration layer makes regular eligibility backend-neutral inside `SnapshotPresentation`. +The top-level viewport carrier survives. +Every other node needs an interactive accessibility type or a non-empty label, identifier, or value. +Hittability no longer admits an otherwise ineligible node. +Raw membership remains unchanged. +This is runner eligibility, not daemon publication membership. +Backend-blind daemon compaction retains ownership of its declared noise suppressions. +If eligibility removes a structural wrapper, presentation reparents its surviving descendants to the nearest surviving ancestor. +Presentation also normalizes their indexes and depths. + +The second semantic layer makes scope a presentation specification instead of an acquisition or daemon-compaction policy. +A trimmed non-empty scope selects the first presentation-preorder match whose subtree contributes to the requested projection. +The match examines label, identifier, and value case-insensitively. +The selected subtree is re-rooted. +Depth applies relative to that root. +If no node matches, presentation publishes an empty healthy projection. +`contracts/fixtures/snapshot-scope-policy.json` pins the Swift and TypeScript implementations. +Scoped iOS acquisition stays broad (including when depth is requested) until an adapter can prove a narrowing hint complete. +The daemon never reapplies scope after the wire. +Android selects its root inside its TypeScript presentation. +Desktop surface runtimes retain their platform projection. The third semantic layer splits presentation into two projections and gives acquisition one input. -`SnapshotPresentation.captureHint` derives a `CaptureHint` from the request; backends read the hint, -never `PresentationOptions`. A hint names the projection the acquisition must serve and may narrow -acquisition only where the backend can prove the narrowing complete for that projection: scope and -its relative depth never narrow, raw depth does (raw depth *is* traversal depth), and the raw -projection never carries `interactiveOnly`. `presentRegular` folds visibility, eligibility, scope, -and scroll hints; `presentRaw` is the acquired tree, normalized, with scope and depth applied only -when the request asked for them — so `interactive ⊆ regular ⊆ raw` holds per backend rather than per -backend implementation. `snapshot --raw -i` therefore returns the acquired tree instead of an -interactive-filtered one. - -Two structural rules keep a backend from answering a request with the other projection, the shape -that let a recovered `snapshot --raw` return viewport-pruned nodes labeled raw: the raw diagnostic -plan is derived from `SnapshotBackendKind.supportsRawProjection` rather than hand-listed, so a -backend with no hierarchy to return (the query sweep) cannot be planned for raw; and presentation -compares the requested projection with the hint the acquisition was captured under, dropping that -tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure instead of presenting it under the -requested label. - -The fourth semantic layer moves the clip fold itself into presentation. Acquisition backends are -fact serializers: every traversed node is emitted at raw traversal depth with its reported frame, -and `SnapshotAcquisition` carries the viewport. The fold returns a typed carrier with both values: -`raw.rect` remains runner-internal reported geometry, while regular presentation writes the -carrier's effective rectangle through the existing wire `rect` field; raw projections and direct -single-element reads retain reported geometry. `presentRegular` runs the one visibility -interpreter for every backend — viewport ∩ scroll-container clip, the ancestor projection cursor -(an out-of-clip Cell or scroll container hides descendants whose clamped frames would otherwise -leak back into the viewport), the sub-pixel decoration rule, hidden-content hints booked onto -scroll anchors, and reparenting of survivors with collapsed depth. The fold also narrows the -emitted `hittable` to the clip: nothing outside its clip, and nothing without geometry, is ever -hittable regardless of what a backend reported. Platform differences are a `SnapshotFoldPolicy` -input to the shared algorithm (iOS cursor-projected; macOS/tvOS plain viewport intersection), -never a backend exception. The presentation owner validates every framed regular node against its -cumulative effective clip before constructing `SnapshotPresentation.PresentedNode`; frameless and -degenerate semantic carriers stay -eligible but are never actionable, while raw projection remains exempt by contract. A violation is -a typed `IOS_SNAPSHOT_PRESENTATION_FAILED` capture failure with the named `presentation-failed` -snapshot-quality reason, preserved through recovery and the existing TypeScript verdict/warning -contract. - -Inside the runner the viewport is a declared fact, not a rectangle: `SnapshotViewport` is -`reported(box, interfaceOrientation)`, `derived(box)`, or `missing(reason)`, the cases of the host's -`IosViewportEvidence` (#2891). Only `reported` carries an orientation, so only it can anchor a -rotation in `SnapshotGeometrySpace`. With no box the clip skips, the cumulative-clip invariant has no -root clip to violate, and a node whose actionability depends on containment has no `hittable` on the -wire, as on the host bridge; disabled or degenerate nodes stay declared `false`. A rectangle becomes a -`Box` only through the initializer that checks it, and the shared guard refuses `CGRect.infinite` by -identity — its components and its extents are all finite, so no comparison would have caught the -value a failed read leaves behind. The two twins were not twins before #2908 landed: the Swift guard -already refused that value by identity, while the TypeScript one accepted it and both accepted finite -components whose right or bottom edge overflowed. Both now refuse both shapes, and every box either -guard accepted and is neither of those two shapes is still classified the same way. The runner route's -host evidence comes from the payload's root nodes (`resolveIosViewportEvidenceFromRoots` in -`packages/capture-kit/src/ios-snapshot-acquisition.ts`). `contracts/fixtures/snapshot-actionability-policy.json` -pins the predicate for shapes the 320x240 fold fixture cannot reach. - -A regular `--depth` request is a presentation cut, not an acquisition bound. `CaptureHint` keeps raw -traversal depth (`--raw --depth`) separate from regular presented depth, but the recursive tree walk -no longer reads presented depth (or any geometry) while descending: for a regular capture it -enumerates the hierarchy — a runaway node cap bounds the walk (#1105, #1156) — and serializes each -node at reported traversal depth and the frame the platform reported. #2661 also removed the per-node -coordinate space (`geometrySpace` / `parentIsWindow`) that #2612 had threaded through every walker; -the one coordinate-space decision is now a single post-acquisition pass over the flat array -(`SnapshotGeometrySpace.normalized`, run in `captureWithBackend`) that keys on ancestry instead of a -value carried down the stack. That pass is why the earlier visible-depth frontier had to go: the -frontier consulted the shared fold mid-walk, and a post-walk normalization pass cannot feed a decision -the walk has already taken — the fold would read reported geometry, and a turned keyboard band in the -device's native space could be cut at the wrong presented depth before the pass ever ran. With the -frontier gone, no acquisition-time decision reads geometry, the walk is bounded only by raw traversal -depth and the node cap, and the visibility fold and the presented-depth cut both happen inside -`SnapshotPresentation`, on the normalized array, at `maximumDepth`. The frontier only ever engaged -when a presented depth was set (`snapshot --depth N`; `snapshot -i` carries none), so that is the -route whose cost the change could move. Measured there on the recursive tier, 15 warm captures per -cell against `main` on the same simulator: the form (135 raw nodes, 4 or 6 presented) and the -scrolled catalog list (279 raw nodes, 4 or 6 presented) walk their whole raw tree on this branch -where `main` pruned it at the presented depth, and acquisition p50 stays within ±4 % and p95 within -±7 % of `main` with mixed sign. The walk runs over an `XCUIElementSnapshot` tree the platform has -already materialized in one call, which is where acquisition time goes; the node construction the -frontier saved is not measurable on these trees. Screens that recover to private AX are untouched -by it. The bound that remains is the raw node cap. De-duplication drops a repeated node and re-parents its children onto that node's -own parent, so identical rows collapse under one addressable owner instead of splitting a subtree -across two nodes with the same identity. Scoped captures remain broad because depth is relative to the -scope root selected in presentation. - -Backend capability declarations are part of the contract, and they describe how much acquisition -work a regular depth request bounds — never whether the backend may answer it. Every backend -serves a regular `--depth` request because presentation applies the presented-depth cut to -whatever hierarchy was acquired: the recursive tree and private AX both enumerate their hierarchy -and are cut afterwards (`presentation-cut`), and the flat query sweep has only its root and one -presented level (so a cut past depth 1 returns the sweep unchanged, `flat`). Completeness below an -acquisition cap is disclosed the same way it is for an -unscoped capture — through `truncated` and `effectiveDepth` — because a depth-capped regular -capture is a subset of the unscoped one from the same backend. Refusing the request instead -produced no answer at all: a plan pinned or deferred to private AX fell through to the synthetic -sparse root, which the daemon then rejected as a missing viewport (#2403). Raw depth remains -acquisition depth for every backend. - -Acquisition-side limits remain explicit: raw private-AX captures still disclose their bridge-side -node cap, the flat query sweep still drops frameless elements because it has no hierarchy to attach -geometryless semantics to, and the recursive tree still has no raw-depth extension for deep XCTest -trees. Presentation cannot repair any of those acquisition limits. - -When adding new iOS snapshot behavior, maintainers should first decide which strategy owns it. If a -change tries to make regular snapshots fast by dropping visible controls behind a node budget, or -tries to make raw snapshots safe by silently truncating, it is probably crossing strategy -boundaries. +`SnapshotPresentation.captureHint` derives a `CaptureHint` from the request. +Backends read the hint, never `PresentationOptions`. +A hint names the projection that the acquisition must serve. +A hint may narrow acquisition only where the backend can prove the narrowing complete for that projection. +Scope and its relative depth never narrow acquisition. +Raw depth does narrow it, because raw depth *is* traversal depth. +The raw projection never carries `interactiveOnly`. +`presentRegular` folds visibility, eligibility, scope, and scroll hints. +`presentRaw` is the acquired tree, normalized. +It applies scope and depth only when the request asked for them. +Thus, `interactive ⊆ regular ⊆ raw` holds per backend, not per backend implementation. +`snapshot --raw -i` therefore returns the acquired tree instead of an interactive-filtered tree. + +Two structural rules keep a backend from answering a request with the other projection. +An answer with the other projection is the shape that let a recovered `snapshot --raw` return viewport-pruned nodes labeled raw. +The first rule derives the raw diagnostic plan from `SnapshotBackendKind.supportsRawProjection`, not from a manual list. +Thus, a backend with no hierarchy to return (the query sweep) cannot be planned for raw. +In the second rule, presentation compares the requested projection with the hint that the acquisition was captured under. +If they differ, presentation drops that tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure. +It does not present that tier under the requested label. + +The fourth semantic layer moves the clip fold itself into presentation. +Acquisition backends are fact serializers. +They emit every traversed node at raw traversal depth with its reported frame. +`SnapshotAcquisition` carries the viewport. +The fold returns a typed carrier with both values: reported geometry and effective geometry. +`raw.rect` remains runner-internal reported geometry. +Regular presentation writes the carrier's effective rectangle through the existing wire `rect` field. +Raw projections and direct single-element reads retain reported geometry. + +For every backend, `presentRegular` runs the one visibility interpreter, which includes: + +- Viewport ∩ scroll-container clip. +- The ancestor projection cursor. +- The sub-pixel decoration rule. +- Hidden-content hints booked onto scroll anchors. +- Reparenting of survivors with collapsed depth. + +With the cursor, an out-of-clip Cell or scroll container hides descendants whose clamped frames would otherwise leak back into the viewport. +The fold also narrows the emitted `hittable` to the clip. +Nothing outside its clip, and nothing without geometry, is ever hittable, regardless of what a backend reported. +Platform differences are a `SnapshotFoldPolicy` input to the shared algorithm, never a backend exception. +iOS is cursor-projected. +macOS/tvOS use plain viewport intersection. + +Before it constructs `SnapshotPresentation.PresentedNode`, the presentation owner validates every framed regular node against its cumulative effective clip. +Frameless and degenerate semantic carriers stay eligible but are never actionable. +Raw projection remains exempt by contract. +A violation is a typed `IOS_SNAPSHOT_PRESENTATION_FAILED` capture failure with the named `presentation-failed` snapshot-quality reason. +Recovery and the existing TypeScript verdict/warning contract preserve that failure. + +Inside the runner, the viewport is a declared fact, not a rectangle. +`SnapshotViewport` has the cases of the host's `IosViewportEvidence` (#2891): + +- `reported(box, interfaceOrientation)`. +- `derived(box)`. +- `missing(reason)`. + +Only `reported` carries an orientation, so only it can anchor a rotation in `SnapshotGeometrySpace`. +If the viewport has no box, the clip is skipped, and the cumulative-clip invariant has no root clip to violate. +If the viewport has no box, a node whose actionability depends on containment has no wire `hittable`, as on the host bridge. +Disabled or degenerate nodes stay declared `false`. +A rectangle becomes a `Box` only through the initializer that checks it. +The shared guard refuses `CGRect.infinite` by identity. +The components and the extents of that value are all finite, so no comparison would have caught the value that a failed read leaves behind. +The Swift guard and the TypeScript guard both refuse two shapes: `CGRect.infinite`, and finite components whose right or bottom edge overflows. +Every box that either guard accepted before #2908, and that is neither of those two shapes, is still classified the same way. +The runner route's host evidence comes from the payload's root nodes (`resolveIosViewportEvidenceFromRoots` in `packages/capture-kit/src/ios-snapshot-acquisition.ts`). +`contracts/fixtures/snapshot-actionability-policy.json` pins the predicate for shapes that the 320x240 fold fixture cannot reach. + +A regular `--depth` request is a presentation cut, not an acquisition bound. +`CaptureHint` keeps raw traversal depth (`--raw --depth`) separate from regular presented depth. +However, the recursive tree walk no longer reads presented depth (or any geometry) while it descends. +For a regular capture, it enumerates the hierarchy, and a runaway node cap bounds the walk (#1105, #1156). +It serializes each node at reported traversal depth with the frame that the platform reported. +The change in #2661 also removed the per-node coordinate space (`geometrySpace` / `parentIsWindow`) that #2612 had threaded through every walker. +The one coordinate-space decision is now a single post-acquisition pass over the flat array. +That pass (`SnapshotGeometrySpace.normalized`, run in `captureWithBackend`) keys on ancestry instead of a value carried down the stack. + +That pass is why the earlier visible-depth frontier was removed. +The frontier consulted the shared fold during the walk. +A post-walk normalization pass cannot feed a decision that the walk has already taken. +The fold would read reported geometry. +Thus, a turned keyboard band in the device's native space could be cut at the wrong presented depth before the pass ran. +With the frontier gone, no acquisition-time decision reads geometry. +The walk is bounded only by raw traversal depth and the node cap. +The visibility fold and the presented-depth cut both happen inside `SnapshotPresentation`, on the normalized array, at `maximumDepth`. + +The frontier only engaged when a presented depth was set (`snapshot --depth N`). +`snapshot -i` carries no presented depth. +Thus, `snapshot --depth N` is the route whose cost the change could move. +Measurements on that route used the recursive tier, 15 warm captures per cell, against `main` on the same simulator: + +- Form: 135 raw nodes, 4 or 6 presented. +- Scrolled catalog list: 279 raw nodes, 4 or 6 presented. +- Acquisition p50: within ±4 % of `main`, with mixed sign. +- Acquisition p95: within ±7 % of `main`, with mixed sign. + +On the measured branch, both screens walk their whole raw tree, where `main` pruned it at the presented depth. +The walk runs over an `XCUIElementSnapshot` tree that the platform has already materialized in one call, which is where acquisition time goes. +The node construction that the frontier saved is not measurable on these trees. +Screens that recover to private AX are not affected. +The bound that remains is the raw node cap. +De-duplication drops a repeated node and re-parents its children onto that node's own parent. +Thus, identical rows collapse under one addressable owner instead of a subtree split across two nodes with the same identity. +Scoped captures remain broad because depth is relative to the scope root selected in presentation. + +Backend capability declarations are part of the contract. +They describe how much acquisition work a regular depth request bounds, never whether the backend may answer it. +Every backend serves a regular `--depth` request because presentation applies the presented-depth cut to whatever hierarchy was acquired. +The recursive tree and private AX both enumerate their hierarchy and are cut afterwards (`presentation-cut`). +The flat query sweep has only its root and one presented level. +Thus, a cut past depth 1 returns the sweep unchanged (`flat`). +Completeness below an acquisition cap is disclosed in the same way as for an unscoped capture, through `truncated` and `effectiveDepth`, because a depth-capped regular capture is a subset of the unscoped capture from the same backend. +A refusal of the request instead produced no answer at all. +A plan pinned or deferred to private AX fell through to the synthetic sparse root, which the daemon then rejected as a missing viewport (#2403). +Raw depth remains acquisition depth for every backend. + +Acquisition-side limits remain explicit: + +- Raw private-AX captures still disclose their bridge-side node cap. +- The flat query sweep still drops frameless elements because it has no hierarchy to attach geometryless semantics to. +- The recursive tree still has no raw-depth extension for deep XCTest trees. + +Presentation cannot repair any of those acquisition limits. + +When maintainers add new iOS snapshot behavior, they should first decide which strategy owns it. +A change probably crosses strategy boundaries if it tries to make regular snapshots fast by dropping visible controls behind a node budget. +A change also probably crosses strategy boundaries if it tries to make raw snapshots safe by silently truncating them. ## Amendment: in-place system surfaces (issue #2438) -Some UI is presented out of the app's process by a system bundle — `com.apple.SafariViewService`, -which hosts `ASWebAuthenticationSession` and `SFSafariViewController` for delegated OAuth/OIDC -sign-in. Two facts, both verified live on the iOS 26.2 Simulator, shape how it is captured: - -- The surface dies if activated. `XCUIApplication.activate()` or `simctl launch` on the host cancels - the authentication session and blacks the view. So the host must be observed and driven **in - place**, never activated, and `open` refuses to launch a registered host. -- The local host AX bridge cannot see it. While the sheet is up the app remains the AX `primaryApp`, - so the bridge serves the (occluded) app tree as if healthy. Only the XCTest runner, addressing the - host by bundle id, can read and drive the sheet. - -Decision. A closed registry names these hosts (`contracts/fixtures/ios-system-surface-hosts.json`, -mirrored by the TypeScript and Swift registries under a parity test). When a registered host is -genuinely presented, the runner serves and drives it in place and never adopts it as the cached -session target; the session binding stays on the app, so once the surface is gone the next command -resolves back to the app. On the Simulator a cheap, device-scoped host-side probe (a registered -host process running for the device) routes the capture to the runner instead of the bridge; when no -host is running the bridge fast path is untouched. - -A presented surface also outranks an explicitly requested bundle id: the runner checks for a -presented host before it resolves or activates `command.appBundleId`, so a command that names a -*different* app is still served the sheet. That is deliberate — the sheet occludes the screen, so -the named app has nothing readable under it, and the capture discloses which surface it describes — -and it costs nothing once the sheet is gone, because the session binding never moved. - -Presence is `XCUIApplication.state == .runningForeground`, not tree content. The live spike showed a -torn-down host still serving a *richer* tree than a live one, so content heuristics cannot separate -live from dead; foreground state can. Crucially, the only way a host is foreground with a stale tree -is if it was activated or relaunched — which the open guard and the in-place policy both refuse — so -this fix and the never-activate guard are one design: the guard is what makes the foreground -predicate sound. This also makes issue #2438's second bug (a stale tree served confidently after -teardown) unrepresentable for the delegated-auth flow, because the session never binds to the host. - -Captures of a system surface carry a response-level `systemSurface` provenance and the shared -`iosSystemSurfaceDisclosure`, worded per host kind, so the agent is told the controls belong to a -system sheet (web sign-in, Apple Pay) rather than the app. They are also lineaged to the host rather -than the app, so their comparison identity differs from an app capture's by construction: every -consumer that asks "are these two captures the same presentation" refuses a cross-surface pair -through ordinary key equality, and no comparison site carries a surface check of its own. Physical -devices always use the runner, so the in-place serve applies there without a route change; the -Simulator route probe is the only Simulator-specific piece. - -The Apple Pay host (`com.apple.PassbookUIService`) joined the registry for text entry as much as for -snapshots. Its billing, shipping, and contact forms hold text fields the session app's tree cannot -resolve, and a bare `type` addressed to the app process never sees that keyboard. Addressing the -host in place is what lets the runner's first-responder route type into them; no text-entry branch -changed for it. +A system bundle presents some UI outside the app's process. +For example, `com.apple.SafariViewService` hosts `ASWebAuthenticationSession` and `SFSafariViewController` for delegated OAuth/OIDC sign-in. +Two facts, both verified live on the iOS 26.2 Simulator, decide how such UI is captured: + +- **The surface dies if activated.** `XCUIApplication.activate()` or `simctl launch` on the host cancels the authentication session and makes the view black. + Thus, the host must be observed and operated **in place**, never activated. + `open` refuses to launch a registered host. +- **The local host AX bridge cannot see the surface.** While the sheet is up, the app remains the AX `primaryApp`. + Thus, the bridge serves the (occluded) app tree as if it were healthy. + Only the XCTest runner, which addresses the host by bundle id, can read and operate the sheet. + +Decision. A closed registry names these hosts (`contracts/fixtures/ios-system-surface-hosts.json`). +The TypeScript and Swift registries mirror it under a parity test. +When a registered host is genuinely presented, the runner serves and operates it in place. +The runner never adopts the host as the cached session target. +The session binding stays on the app. +Thus, after the surface is gone, the next command resolves back to the app. +On the Simulator, a cheap, device-scoped host-side probe checks for a registered host process that runs for the device. +If such a host is running, the probe routes the capture to the runner instead of the bridge. +If no host is running, the bridge fast path is unchanged. + +A presented surface also takes precedence over an explicitly requested bundle id. +The runner checks for a presented host before it resolves or activates `command.appBundleId`. +Thus, a command that names a *different* app still receives the sheet. +This is intentional. +The sheet occludes the screen, so the named app has nothing readable under it, and the capture discloses which surface it describes. +After the sheet is gone, this costs nothing, because the session binding never moved. + +Presence is `XCUIApplication.state == .runningForeground`, not tree content. +The live experiment showed a torn-down host that still served a *richer* tree than a live host. +Thus, content heuristics cannot separate a live host from a dead host, but foreground state can. +A host can be foreground with a stale tree only if it was activated or relaunched. +The open guard and the in-place policy both refuse those operations. +Thus, this fix and the never-activate guard are one design: the guard makes the foreground predicate sound. +Because the session never binds to the host, issue #2438's second bug cannot occur for the delegated-auth flow. +That bug was a stale tree served confidently after teardown. + +Captures of a system surface carry a response-level `systemSurface` provenance and the shared `iosSystemSurfaceDisclosure`, worded per host kind. +Thus, the agent is told that the controls belong to a system sheet (web sign-in, Apple Pay) instead of the app. +These captures are also lineaged to the host instead of the app. +Thus, their comparison identity differs from an app capture's identity by construction. +Every consumer that asks "are these two captures the same presentation" refuses a cross-surface pair through ordinary key equality. +No comparison site carries a surface check of its own. +Physical devices always use the runner, so the in-place serve applies there without a route change. +The Simulator route probe is the only Simulator-specific piece. + +The Apple Pay host (`com.apple.PassbookUIService`) joined the registry for text entry as much as for snapshots. +Its billing, shipping, and contact forms hold text fields that the session app's tree cannot resolve. +A bare `type` addressed to the app process never sees that keyboard. +When the runner addresses the host in place, its first-responder route can type into those fields. +No text-entry branch changed for it. ## Amendment: the coordinate space of a captured subtree (issue #2612) -Some system surfaces keep their geometry in the device's native (portrait-up) space while the app -is rotated, and their *whole subtree* arrives turned with them. Measured on iPhone 17 Pro (iOS -26.2) with the system keyboard up over a landscape app frame `(0,0,874,402)`: -`UIRemoteKeyboardWindow` reports its own box as `(0,0,402,874)` — the app's box with its two side -lengths swapped — while the app's own window and `UITextEffectsWindow` both report `(0,0,874,402)`. -In portrait the two spaces coincide and nothing is turned. Nothing but that box says which space a -subtree reports in, and a consumer reading the numbers cannot tell a keyboard laid across the -bottom of a landscape screen from a strip running down its left edge: it refused app content the -keyboard was nowhere near and let a tap land on a key. - -Decision. One published capture publishes one space — the app's orientation space — and the -producer settles it where the platform's frame is still the platform's, because downstream of the -capture the question is unanswerable: a merged tree carries no axis a reader could turn geometry -against. A window declares the space of its own subtree from its own box, which is either the app's -box or that box turned through a quarter, and anything that is not a window inherits the space of -the window above it. A published rect and the actionability verdict read from it come off the same -oriented frame, so an address and the permission to tap it cannot disagree. Where no space can be -named — no usable app frame, an interface orientation the platform did not report, or an app frame -square enough to be indistinguishable from its own quarter turn — the capture publishes what the -platform reported rather than guessing at a rotation, and consumers fail open on that geometry the -way they do on any missing platform fact. Detection and the way back share one rotation table with -synthesized dispatch — pure geometry that lives in the `AgentDeviceSnapshotPresentation` package -(`SnapshotCoordinateSpace.swift`), not in the XCTest bundle — and both languages replay it against -`contracts/fixtures/window-coordinate-space.json` in the ADR 0011 parity-table shape, which is what -stops capture from turning back with anything other than the exact inverse of what dispatch turns -forward. - -Decision. A producer that cannot name the app's interface orientation refuses the screen rather -than publishing two spaces in one tree. The Simulator AX bridge cannot name it: the #2659 spike -(verdict on that issue, write-up in the diff of #2667) measured that the one AX attribute for it, -`XC_kAXXCAttributeApplicationOrientation` (id 1503), resolves but reads 0 through the guest's -snapshot channel and errors `kAXErrorServerNotFound` through XCTest's own reader, and that the only -cheap service read is *device* orientation, which diverges from the app's on a rotation-locked app. -So its decoder counts window roots reporting the app's box quarter-turned and -fails that capture (`window-coordinate-space-unresolved`, kind `unsupported`) — the same refusal -shape as `remote-content-boundary` above — and the route serves the runner, which reads the -orientation. A mixed tree is not half-usable: a published rect is an address, and once some rects -in a tree answer to a turned axis no pair of them answers "how far apart are these" any more, so -every consumer downstream would have to know which windows to distrust — a rule the capture could -have applied and did not. The refusal costs one runner round trip and is correct. - -Decision. That refusal is scoped to the capture, not to the app generation. Entering the generation -circuit is for failures that are evidence about the app: a web-view screen is a property of a -hybrid app, so its screens keep the runner until it relaunches. A rotated surface is evidence about -the screen in front of the reader and about nothing else — it is up now and gone after the next -keystroke — so retiring the generation would move every later portrait capture of a healthy app -onto the runner to work around one landscape keyboard, the cost #2491 settled for a bridge that was -merely still being prepared. Which side of that line a failure falls on is a declared property of -its code rather than a judgment made at the call site. - -What stays unnormalized, and why the consumer rule stays: the runner's recursive-tree capture now -answers the keyboard with a measured band instead of a rebuilt one, because the same run that owns -the tree can ask `app.keyboards` and gets an answer in the app's own orientation space (#2660). That -closes the keyboard question on the path that used to lean on this ADR's geometry most, and it is why -the guard's landscape refusals no longer depend on whether the tree arrived turned. - -What did not move is the rest of the sentence. Capture settles the space in ONE pass over the flat -acquired array (`SnapshotGeometrySpace.normalized`, run once in `captureWithBackend` between -acquisition and presentation, #2661), keyed on each node's `type`, `parentIndex` and `rect`. The -runner's query-sweep tier still has no window ancestry, so nothing in its tree declares a native space: -its flat children hang off a synthetic application root that reports the app's own box under an -interface orientation the tier does not read, so the pass returns that tree exactly as reported — a -structural consequence of the same rule applied uniformly, not the per-tier omission a hand-threaded -space would have made it. The provider producers (`appium-source`, `limrun-ios-tree`) never see the -app's windows either; a capture from any of them publishes no keyboard fact, and a consumer reads that -silence as "this producer did not measure". Those paths publish what the platform reported, so the -last reader that can still refuse geometry it cannot place is the tap-path keyboard guard, and its -width rule therefore remains. The rule detects un-normalized -arrival, not a standing fact about iOS: the producers above do normalize, and a band taller than it -is wide is what one that did not looks like. +Some system surfaces keep their geometry in the device's native (portrait-up) space while the app is rotated. +Their *whole subtree* arrives turned with them. +A measurement used iPhone 17 Pro (iOS 26.2) with the system keyboard up over a landscape app frame `(0,0,874,402)`: + +- `UIRemoteKeyboardWindow` reports its own box as `(0,0,402,874)`, which is the app's box with its two side lengths swapped. +- The app's own window and `UITextEffectsWindow` both report `(0,0,874,402)`. + +In portrait, the two spaces coincide and nothing is turned. +Nothing but that box says which space a subtree reports in. +From the numbers alone, a consumer cannot tell a keyboard across the bottom of a landscape screen from a strip down its left edge. +Thus, the consumer refused app content far from the keyboard and let a tap land on a key. + +Decision. One published capture publishes one space: the app's orientation space. +The producer settles the space where the platform's frame is still the platform's frame. +The reason is that downstream of the capture, the question cannot be answered: a merged tree carries no axis that a reader could turn geometry against. +A window declares the space of its own subtree from its own box. +That box is either the app's box or that box turned through a quarter. +Anything that is not a window inherits the space of the window above it. +A published rect and the actionability verdict read from it come from the same oriented frame. +Thus, an address and the permission to tap it cannot disagree. + +In some cases, no space can be named: + +- No usable app frame exists. +- The platform did not report an interface orientation. +- The app frame is square enough to be indistinguishable from its own quarter turn. + +In these cases, the capture publishes what the platform reported instead of guessing a rotation. +Consumers fail open on that geometry, as they do on any missing platform fact. +Detection and the way back share one rotation table with synthesized dispatch. +That table is pure geometry that lives in the `AgentDeviceSnapshotPresentation` package (`SnapshotCoordinateSpace.swift`), not in the XCTest bundle. +Both languages replay it against `contracts/fixtures/window-coordinate-space.json` in the ADR 0011 parity-table shape. +This replay stops capture from turning back with anything other than the exact inverse of what dispatch turns forward. + +Decision. A producer that cannot name the app's interface orientation refuses the screen instead of publishing two spaces in one tree. +The Simulator AX bridge cannot name that orientation. +The #2659 spike (verdict on that issue, write-up in the diff of #2667) measured these facts: + +- The one AX attribute for it, `XC_kAXXCAttributeApplicationOrientation` (id 1503), resolves but reads 0 through the guest's snapshot channel. +- Through XCTest's own reader, that attribute errors with `kAXErrorServerNotFound`. +- The only cheap service read is *device* orientation, which diverges from the app's orientation on a rotation-locked app. + +Thus, the bridge's decoder counts window roots that report the app's box quarter-turned. +It fails that capture (`window-coordinate-space-unresolved`, kind `unsupported`). +This is the same refusal shape as `remote-content-boundary` above. +The route then serves the runner, which reads the orientation. +A mixed tree is not half-usable. +A published rect is an address. +After some rects in a tree answer to a turned axis, no pair of rects in that tree can answer "how far apart are these" any more. +Thus, every downstream consumer would have to know which windows to distrust. +The capture could have applied that rule and did not. +The refusal costs one runner round trip and is correct. + +Decision. That refusal is scoped to the capture, not to the app generation. +The generation circuit is for failures that are evidence about the app. +A web-view screen is a property of a hybrid app, so its screens keep the runner until the app relaunches. +A rotated surface is evidence about the screen in front of the reader and about nothing else. +It is up now and gone after the next keystroke. +If the refusal retired the generation, every later portrait capture of a healthy app would move to the runner because of one landscape keyboard. +That is the cost that #2491 settled for a bridge that was only still in preparation. +Which side of that line a failure falls on is a declared property of its code, not a judgment made at the call site. + +The rest of this section states what stays unnormalized, and why the consumer rule stays. +The runner's recursive-tree capture now answers the keyboard with a measured band instead of a rebuilt band. +The reason is that the same run that owns the tree can ask `app.keyboards` and gets an answer in the app's own orientation space (#2660). +This closes the keyboard question on the path that used to depend most on this ADR's geometry. +It is also why the guard's landscape refusals no longer depend on whether the tree arrived turned. + +Capture still settles the space in ONE pass over the flat acquired array. +`SnapshotGeometrySpace.normalized` runs once in `captureWithBackend`, between acquisition and presentation (#2661). +The pass keys on each node's `type`, `parentIndex`, and `rect`. +The runner's query-sweep tier still has no window ancestry, so nothing in its tree declares a native space. +Its flat children hang off a synthetic application root that reports the app's own box. +That root reports the box under an interface orientation that the tier does not read. +Thus, the pass returns that tree exactly as reported. +This is a structural consequence of the same rule applied uniformly. +It is not the per-tier omission that a hand-threaded space would have made it. + +The provider producers (`appium-source`, `limrun-ios-tree`) never see the app's windows either. +A capture from any of them publishes no keyboard fact. +A consumer reads that silence as "this producer did not measure". +Those paths publish what the platform reported. +Thus, the last reader that can still refuse geometry that it cannot place is the tap-path keyboard guard, and its width rule therefore remains. +The rule detects un-normalized arrival, not a standing fact about iOS. +The producers that the earlier sections of this ADR describe do normalize. +A band taller than it is wide is what the output of a producer that did not normalize looks like. ## Amendment: modal containment in the presentation cut -The bridge reads one window and reports every container hanging off it, while XCTest's own queries -answer only the presentation the user can reach. React Navigation's card-plus-modal example is the -measured case: three routes presented as sheets left three transition-view containers in one window -and one capture published 567 nodes across three screens, where the same app state under the runner -published 76 across the top screen alone. Maestro's visibility test is geometric, so a `tapOn` -resolved the covered screen's button and the flow failed on an assertion about the screen that never -arrived, and a `presentation: "formSheet"` route's fields matched intermittently (#2638). - -Decision. The fold applies modal containment: when the last transition view under a container carries -UIKit's dimming view as its direct child and the producer reports that dimming view takes touches -(`userInteractionEnabled: true`), the earlier transition views whose frame that dimmed area spans are -cut, with their subtrees, from the regular and interactive projections. Every part of the claim is a -producer fact rather than an ordering guess — the dimming view is UIKit's own declaration that it dims -what sits behind, its interaction state says whether a touch there reaches what sits behind, and its -frame is the producer's rectangle, which is what a covered container must be inside. A presentation -with no dimming view, a dimming view the producer did not read or reports passing touches through, -and any container whose earlier sibling is not a transition view keep today's behavior: containment -is asserted only where the producer states it, so the rule fails closed rather than guessing which -siblings are shadows. - -A sheet resting at an undimmed detent (`largestUndimmedDetentIdentifier`, react-native-screens -`sheetLargestUndimmedDetentIndex`) is why the interaction state is part of the claim. UIKit keeps the -dimming view in the tree at the same window-sized frame, and the presenting screen stays reachable: on -react-navigation's form-sheet example a press on the presenting screen's `Height Steps` button -navigates while the `Custom Dimming` sheet rests at its smallest detent. The bridge reports that -dimming view `userInteractionEnabled: false`, and `true` for the dimmed form sheet and the card modal. -No attribute the bridge already read told them apart, and `XC_kAXXCAttributeIsVisible` cannot either: -it reads false for the undimmed sheet's dimming view and for the card modal's. The bridge reads the -attribute for dimming views alone, in one follow-up read per view: requesting it on every node cost -about half again the capture time on a 492-node tree, while the targeted read did not move it. -Every source the cut removes is counted in the presentation's -`stats.modalContainedNodeCount` — internal evidence, per ADR 0026, never wire vocabulary — because -the same screen either side of an animation otherwise moves hundreds of comparable lines with nothing -to attribute them to. - -Why this seam and not the neighbouring ones. The semantic presentation rules -(`IOS_PRESENTATION_RULES`) run only for the interactive projection, so a suppression rule there could -not deliver the same claim to a regular `snapshot`; the regular eligibility table is pinned against -its Swift twin, so widening it would move a cross-language contract that this rule does not touch; the -occlusion pass *marks* a node covered and keeps it published, and a published-but-marked node is -exactly the 567-versus-76 divergence being fixed, since visibility filtering reads geometry and not -the mark; and cutting at acquisition would leak the decision into `--raw`, which owes the reader the -tree the platform reported. Raw therefore still carries the covered screens. - -Producers that report no UIKit class names or no dimming-view interaction state never trigger the -cut, and that is a fact about their output rather than a backend exception in the fold: the runner already omits modal-contained content -from its own queries, `appium-source` and `limrun-ios-tree` report element types and no classes, and -the macOS desktop surface arrives already presented. A scope naming a modal-contained screen now -publishes an empty projection, which is the same healthy empty answer any other unmatched scope gives -— the screen is not what is presented. +The bridge reads one window and reports every container attached to it. +XCTest's own queries answer only the presentation that the user can reach. +React Navigation's card-plus-modal example is the measured case. +Three routes presented as sheets left three transition-view containers in one window. +One capture published 567 nodes across three screens. +The same app state under the runner published 76 nodes across the top screen alone. +Maestro's visibility test is geometric. +Thus, a `tapOn` resolved the covered screen's button, and the flow failed on an assertion about the screen that never arrived. +Also, the fields of a `presentation: "formSheet"` route matched intermittently (#2638). + +Decision. The fold applies modal containment when all of these conditions are true: + +- The last transition view under a container carries UIKit's dimming view as its direct child. +- The producer reports that the dimming view takes touches (`userInteractionEnabled: true`). + +Then the fold cuts the earlier transition views whose frame that dimmed area spans. +It cuts them, with their subtrees, from the regular and interactive projections. +Every part of the claim is a producer fact, not a guess from order: + +- The dimming view is UIKit's own declaration that it dims what sits behind it. +- Its interaction state says whether a touch there reaches what sits behind it. +- Its frame is the producer's rectangle, which is what a covered container must be inside. + +These cases keep today's behavior: + +- A presentation with no dimming view. +- A dimming view that the producer did not read. +- A dimming view that the producer reports as passing touches through. +- Any container whose earlier sibling is not a transition view. + +Containment is asserted only where the producer states it. +Thus, the rule fails closed instead of guessing which siblings are shadows. + +A sheet that rests at an undimmed detent is why the interaction state is part of the claim. +UIKit `largestUndimmedDetentIdentifier` and react-native-screens `sheetLargestUndimmedDetentIndex` configure such a detent. +UIKit keeps the dimming view in the tree at the same window-sized frame, and the presenting screen stays reachable. +In react-navigation's form-sheet example, a press on the presenting screen's `Height Steps` button navigates while the `Custom Dimming` sheet rests at its smallest detent. +The bridge reports that dimming view as `userInteractionEnabled: false`. +It reports `true` for the dimmed form sheet and for the card modal. +No attribute that the bridge already read told them apart. +`XC_kAXXCAttributeIsVisible` cannot tell them apart either. +It reads false for the undimmed sheet's dimming view and for the card modal's dimming view. + +The bridge reads the attribute for dimming views alone, in one follow-up read per view. +A request for it on every node increased the capture time by about half on a 492-node tree. +The targeted read did not change the capture time. +Every source that the cut removes is counted in the presentation's `stats.modalContainedNodeCount`. +Per ADR 0026, that count is internal evidence, never wire vocabulary. +The count exists because otherwise the same screen on either side of an animation moves hundreds of comparable lines with nothing to attribute them to. + +The neighboring seams do not fit this rule: + +- The semantic presentation rules (`IOS_PRESENTATION_RULES`) run only for the interactive projection. Thus, a suppression rule there could not deliver the same claim to a regular `snapshot`. +- The regular eligibility table is pinned against its Swift twin. Thus, a wider table would move a cross-language contract that this rule does not touch. +- The occlusion pass *marks* a node covered and keeps it published. Visibility filtering reads geometry, not the mark. Thus, a published-but-marked node is exactly the 567-versus-76 divergence that this rule fixes. +- A cut at acquisition would leak the decision into `--raw`, which owes the reader the tree that the platform reported. + +Raw therefore still carries the covered screens. + +Producers that report no UIKit class names or no dimming-view interaction state never trigger the cut. +That is a fact about their output, not a backend exception in the fold: + +- The runner already omits modal-contained content from its own queries. +- `appium-source` and `limrun-ios-tree` report element types and no classes. +- The macOS desktop surface arrives already presented. + +A scope that names a modal-contained screen now publishes an empty projection. +That is the same healthy empty answer that any other unmatched scope gives, because the screen is not what is presented. diff --git a/docs/adr/0005-ios-runner-interaction-lifecycle.md b/docs/adr/0005-ios-runner-interaction-lifecycle.md index 68f6e0ec2e..a165c91b6a 100644 --- a/docs/adr/0005-ios-runner-interaction-lifecycle.md +++ b/docs/adr/0005-ios-runner-interaction-lifecycle.md @@ -4,149 +4,187 @@ Accepted +## Rules at a glance + +- For iOS/macOS selector taps on an element with a frame, use coordinate-first resolved element activation. tvOS remains focus/remote-driven. +- Before a command to a ready session, send an `uptime` preflight, except for the stated startup, readiness, and recent-mutation cases. +- Keep recent-mutation health only on the `RunnerSession` and only for the stated app, commands, response checks, and 5-second window. +- Never count snapshots or other read-only responses as health signals. +- Future optimization work should reduce the `uptime` preflights only after runner status survives command-induced XCTest teardown and can prove that the session still serves new requests. +- Invalidate dead cached runner processes without graceful `shutdown`. +- After `RUNNER_BUSY`, resend only read-only commands, under their busy-specific budget. +- After `RUNNER_BUSY` on a mutating command, return the refusal to the caller without a resend. +- If `close` would retain a runner whose main-thread occupancy is set, stop that runner first and await the stop. +- After a root AX failure, clear the cached app target. Keep raw/strict errors distinct from interactive recovery payloads. + ## Context -The iOS runner is a long-lived XCTest process with an HTTP command loop. A command can appear to -complete at the daemon boundary while XCTest is already tearing down the test runner. +The iOS runner is a long-lived XCTest process with an HTTP command loop. +A command can appear to complete at the daemon boundary while XCTest is already tearing down the test runner. -This was reproduced in the React Navigation playground with navigation-causing selector taps such -as `Navigate to Details` and `Back to home`. The runner resolved the button and synthesized the tap, -the app navigated, and then XCTest tried to re-resolve the original `XCUIElement`. Because the -element had disappeared, xcodebuild recorded `Failed to get matching snapshot` and ended the test -with `** TEST EXECUTE FAILED **`. The daemon had already received a successful tap response, so the -next read-only command inherited a stale cached runner. +This failure was reproduced in the React Navigation playground with selector taps that cause navigation, such as `Navigate to Details` and `Back to home`. +The runner resolved the button and synthesized the tap. +The app navigated, and XCTest then tried to resolve the original `XCUIElement` again. +Because the element had disappeared, xcodebuild recorded `Failed to get matching snapshot` and ended the test with `** TEST EXECUTE FAILED **`. +The daemon had already received a successful tap response. +Thus, the next read-only command inherited a stale cached runner. -Two older assumptions were wrong: +These earlier assumptions were incorrect: -- A recent successful runner response proves the runner is still healthy. -- `XCUIElement.tap()` is the safest selector-tap primitive once a selector has resolved. -- A cached `XCUIApplication` target remains safe after XCTest reports that the app's accessibility - tree cannot be serialized. +- A recent successful runner response proves that the runner is still healthy. +- After a selector has resolved, `XCUIElement.tap()` is the safest selector-tap primitive. +- A cached `XCUIApplication` target remains safe after XCTest reports that it cannot serialize the app's accessibility tree. ## Decision -Coordinate-first resolved element activation is the iOS/macOS selector-tap model. The runner still -uses selectors or text queries to find the semantic `XCUIElement`, but when the element has a frame, -activation taps the resolved center point instead of calling `XCUIElement.tap()`. tvOS remains -focus/remote-driven because tvOS does not support normal coordinate input. - -Ready runner sessions are probed with a short `uptime` preflight before command send. Read-only -startup commands still skip that preflight because the first successful command is the readiness -proof for a newly launched runner. Readiness probe commands skip preflight to avoid recursion. - -The daemon may additionally skip the ready-session `uptime` preflight for an explicit allowlist of -mutating interactions (`tap`, `longPress`, `drag`, `swipe`, `scroll`, `sequence`) when the same -session produced a healthy mutating response — parsed ok and not carrying `runnerFatal` — for the -same `appBundleId` within 5 seconds. This recency lives only on the `RunnerSession` object as -`lastHealthyMutation`, so it dies with every invalidation/restart, and it is recorded only after the -`runnerFatal` check, so sparse AX-fallback snapshots and `runnerFatal` payloads never refresh it. -Snapshots and other read-only responses never count as a health signal. This narrow skip is -permitted now because the future-work precondition below is met: coordinate-first activation removed -the command-induced teardown trigger, and the lifecycle status journal plus the status-before- -invalidate recovery is the teardown-surviving status surface that resolves any ambiguous post-send -failure before invalidation. A transport failure after a skip clears the recency record and is marked -with the skip context; connection-shaped failures (refused, reset, hung up) run status recovery -instead of a blind replay, while timeout-shaped failures propagate with the skip context (the same -classification preflighted sends use). - -`uptime` is a direct runner listener probe. It is answered before command journaling, the serial -command execution queue, app activation, and main-thread XCTest dispatch. It should measure only -whether the runner is alive and accepting new HTTP requests. - -Dead cached runner processes are invalidated without graceful `shutdown`. A process that already -stopped cannot answer the shutdown request, so graceful cleanup only adds stale-listener delay. - -The runner stamps its live main-thread occupancy (`runnerMainThreadBusy`) onto every successful -response and tags the execution-watchdog timeout with the typed `MAIN_THREAD_TIMEOUT` code, so the -daemon learns the main thread is occupied from the stalling command itself, not only from a later -`RUNNER_BUSY` refusal. The daemon mirrors that occupancy on the `RunnerSession`: set on `RUNNER_BUSY` -or `MAIN_THREAD_TIMEOUT`, cleared by any other served runner reply (which reached the main thread and -therefore drained), and left intact by a transport failure or an unstamped recovered response. A -healthy `ok` is never read as proof of drain because a private-AX snapshot can be served while an -abandoned tree crawl still grinds on the XCTest main thread (#2552). - -What a `RUNNER_BUSY` refusal then costs the caller turns on the daemon's own per-command read-only -trait, not on any runner-side classification of the command: `RUNNER_COMMAND_TRAITS` in -`packages/platform-apple/src/runner/runner-command-traits.ts`, read through -`isReadOnlyRunnerCommand` where `runAppleRunnerCommand` decides whether to wrap the send in a resend -loop. The asymmetry is shipped, and asserted in `runner-client-busy-resend.test.ts`: a read-only -command treats the refusal as *not yet*, waiting the drain out on the busy-specific budget sized to -outlast it and resending until the runner answers or the window ends; a mutating command treats the -same refusal as *not mine to send again* — one attempt, and the refusal reaches the caller unwrapped -with nothing replayed. Neither path pays a status probe, because a structured reply already answered -"did my command run?" (the bypass in `runner-lifecycle.ts`). The same trait already decides the -startup-preflight skip above, so it is the single place a change to that classification lands. - -Close that would retain a runner for reuse first stops it when that occupancy is set, awaiting the -stop so the lease is released before the next request. A runner still finishing watchdog-abandoned -work does refuse every command sent to it until it drains or escalates to `RUNNER_WEDGED`, and that -fact is stated here for the close-and-pooling decision only: the read-only resend above is one live -caller waiting the same refusal out on its own budget, and `close` does not inherit that wait on the -pool's behalf. Pooling the stalled runner back hands it to the next `open` and `close` recovers -nothing. Killing the process is the only way to abort uncancellable XCTest work. This is the -`RUNNER_WEDGED` restart from #1105 applied at the close boundary rather than after the wedge -threshold elapses. - -When XCTest reports a root accessibility snapshot failure such as `kAXErrorIllegalArgument`, the -runner treats the cached app target as suspect. Interactive snapshots fail closed to a truncated -root-only payload instead of issuing more flat fallback queries against the same broken tree, and -the cached `XCUIApplication` handle is cleared so the next command reacquires the target through the -normal activation path. - -An external iOS-family Simulator relaunch or physical iOS relaunch also invalidates process-bound -target state. After replacing the app process, the daemon sends a lifecycle reset to the retained -runner so the next command reacquires `XCUIApplication`; the reset also clears process-bound snapshot -penalty and private-AX depth state. If that reset cannot be confirmed, or if relaunch fails before -reset, the daemon discards the runner session. Other physical Apple OS leaves retain their existing -runner-restart behavior until separately evidenced. - -The snapshot surface intentionally has two AX-failure shapes. Interactive fast snapshots return a -truncated success payload with `runnerFatal` so agents can still see that AX state is unavailable -and recover with a plain screenshot plus coordinate navigation. Raw or strict snapshot paths keep -returning an error because those callers requested a faithful tree, not a lossy recovery payload. +Coordinate-first resolved element activation is the iOS/macOS selector-tap model. +The runner still uses selectors or text queries to find the semantic `XCUIElement`. +If the element has a frame, activation taps the resolved center point instead of calling `XCUIElement.tap()`. +tvOS remains focus/remote-driven because tvOS does not support normal coordinate input. + +Before command send, ready runner sessions receive a short `uptime` preflight. +Read-only startup commands still skip that preflight because, for a newly launched runner, the first successful command is the readiness proof. +Readiness probe commands skip preflight to avoid recursion. + +The daemon may also skip the ready-session `uptime` preflight for an explicit allowlist of mutating interactions: + +- `tap`. +- `longPress`. +- `drag`. +- `swipe`. +- `scroll`. +- `sequence`. + +The skip requires that the same session produced a healthy mutating response for the same `appBundleId` within 5 seconds. +A healthy mutating response is parsed ok and does not carry `runnerFatal`. +This recency lives only on the `RunnerSession` object as `lastHealthyMutation`. +Thus, it is discarded with every invalidation or restart. +The record is written only after the `runnerFatal` check. +Thus, sparse AX-fallback snapshots and `runnerFatal` payloads never refresh it. +Snapshots and other read-only responses never count as a health signal. + +This narrow skip is permitted now, because the preflight-reduction precondition in the Consequences section is met: + +- Coordinate-first activation removed the command-induced teardown trigger. +- The lifecycle status journal plus the status-before-invalidate recovery is the teardown-surviving status surface. + That surface resolves any ambiguous post-send failure before invalidation. + +After a skip, a transport failure clears the recency record and is marked with the skip context. +Connection-shaped failures (refused, reset, hung up) run status recovery instead of a blind replay. +Timeout-shaped failures propagate with the skip context. +This is the same classification that preflighted sends use. + +`uptime` is a direct runner listener probe. +The listener answers it before command journaling, the serial command execution queue, app activation, and main-thread XCTest dispatch. +The `uptime` probe should measure only whether the runner is alive and accepts new HTTP requests. + +Dead cached runner processes are invalidated without graceful `shutdown`. +A process that already stopped cannot answer the shutdown request. +Thus, graceful cleanup only adds stale-listener delay. + +The runner stamps its live main-thread occupancy (`runnerMainThreadBusy`) onto every successful response. +The runner tags the execution-watchdog timeout with the typed `MAIN_THREAD_TIMEOUT` code. +Thus, the daemon learns of main-thread occupancy from the stalling command itself, not only from a later `RUNNER_BUSY` refusal. +The daemon mirrors that occupancy on the `RunnerSession`: + +- `RUNNER_BUSY` or `MAIN_THREAD_TIMEOUT` sets it. +- Any other served runner reply clears it, because that reply reached the main thread and therefore drained. +- A transport failure or an unstamped recovered response leaves it intact. + +The daemon never reads a healthy `ok` as proof of drain. +The reason is that a private-AX snapshot can be served while an abandoned tree crawl still continues on the XCTest main thread (#2552). + +The cost of a `RUNNER_BUSY` refusal to the caller depends on the daemon's own per-command read-only trait. +It does not depend on any runner-side classification of the command. +The trait is `RUNNER_COMMAND_TRAITS` in `packages/platform-apple/src/runner/runner-command-traits.ts`. +`runAppleRunnerCommand` reads it through `isReadOnlyRunnerCommand` when it decides whether to wrap the send in a resend loop. +This asymmetry is shipped, and `runner-client-busy-resend.test.ts` asserts it: + +- A read-only command treats the refusal as *not yet*. + It waits for the drain on the busy-specific budget sized to outlast the drain. + It resends until the runner answers or the window ends. +- A mutating command treats the same refusal as *not mine to send again*. + It makes one attempt, and the refusal reaches the caller unwrapped, with nothing replayed. + +Neither path sends a status probe, because a structured reply already answered "did my command run?". +This bypass is in `runner-lifecycle.ts`. +The same trait already decides the startup-preflight skip above. +Thus, it is the single place for a change to that classification. + +If `close` would retain a runner for reuse and that occupancy is set, `close` first stops the runner. +`close` awaits the stop, so the lease is released before the next request. +A runner that still finishes watchdog-abandoned work refuses every command sent to it until it drains or escalates to `RUNNER_WEDGED`. +This ADR states that fact only for the close-and-pooling decision. +The read-only resend above is one live caller that waits out the same refusal on its own budget. +`close` does not inherit that wait on behalf of the pool. +If the pool takes the stalled runner back, it gives that runner to the next `open`, and `close` recovers nothing. +Killing the process is the only way to abort uncancellable XCTest work. +This is the `RUNNER_WEDGED` restart from #1105, applied at the close boundary instead of after the wedge threshold elapses. + +If XCTest reports a root accessibility snapshot failure such as `kAXErrorIllegalArgument`, the runner treats the cached app target as suspect. +Interactive snapshots fail closed to a truncated root-only payload. +They do not issue more flat fallback queries against the same broken tree. +The runner clears the cached `XCUIApplication` handle. +Thus, the next command reacquires the target through the normal activation path. + +An external iOS-family Simulator relaunch or physical iOS relaunch also invalidates process-bound target state. +After it replaces the app process, the daemon sends a lifecycle reset to the retained runner. +Thus, the next command reacquires `XCUIApplication`. +The reset also clears process-bound snapshot penalty and private-AX depth state. +If that reset cannot be confirmed, or if relaunch fails before reset, the daemon discards the runner session. +Other physical Apple OS leaves retain their existing runner-restart behavior until separate evidence exists. + +The snapshot surface intentionally has two AX-failure shapes. +Interactive fast snapshots return a truncated success payload with `runnerFatal`. +Thus, agents can still see that AX state is unavailable and recover with a plain screenshot plus coordinate navigation. +Raw or strict snapshot paths continue to return an error, because those callers requested a faithful tree, not a lossy recovery payload. ## Consequences -Navigation-causing selector taps no longer couple command success to XCTest's post-tap element -bookkeeping. If the target disappears because navigation happened, the tap remains a normal -successful interaction and the runner should stay alive. - -If xcodebuild still exits for another reason, the next command detects the stale runner through -process/liveness checks and avoids the old 15-second graceful-shutdown wait. The remaining latency is -fresh xcodebuild runner startup, not a stale transport stall. - -A `close` then `open` on a runner still draining abandoned main-thread work now pays a fresh runner -boot instead of inheriting the stalled one, so the wedge no longer survives the close/open cycle. -Runners that are ready and have no unanswered exchanges can remain warm when retention is requested; -iOS-family simulators additionally require an observed live listener. Close retires unfinished -startup and its recovery authority. The retained generation owns its idle timer and listener -observation; successful reuse ends retention, listener loss or idle expiry disposes that generation, -and graceful daemon shutdown stops idle retained generations before exiting. Active ready -generations retain the existing handoff policy. - -The daemon no longer models a generic "recent success" cache as a runner-health signal. A proven -healthy mutating response for the same app — recorded only after the `runnerFatal` check and only -for allowlisted interactions — is now a real end-to-end liveness proof (HTTP listener through to the -app target), so a hot loop of allowlisted interactions skips the per-command `uptime` request while -still re-earning each skip from another healthy mutation. The earlier unconditional `uptime` before -every ready-session command remains the default for non-allowlisted commands and after any -invalidation, stale record, app-bundle change, or absent record. - -Apps with broken accessibility trees may still be impossible for XCTest to inspect deeply, but one -failed snapshot no longer teaches the runner to keep using a suspect cached app target or to amplify -the failure by walking every interactive element query. - -An `XCTEST_RECORDED_FAILURE` returned after a tap is treated as an ambiguous outcome at the daemon -boundary. When a usable retained snapshot exists, the daemon takes one same-presentation post-action -capture. A changed accessibility digest is reported as success with an explicit warning so agents do -not blindly repeat a tap that may already have navigated; unchanged, sparse, mismatched, or unavailable -evidence remains the original failure. - -iOS-family Simulator and physical iOS relaunch keep the healthy XCTest process warm without carrying -an app target across process identity. The reset adds one local runner request instead of paying for a -runner restart and clears the old process's hostile-screen capture penalty before the replacement is -reacquired. - -Future optimization work should only reduce these preflights after the runner exposes status in a -way that survives command-induced XCTest teardown and can prove the session is still serving new -requests. +The success of a selector tap that causes navigation no longer depends on XCTest's post-tap element bookkeeping. +If the target disappears because navigation happened, the tap remains a normal successful interaction. +The runner should stay alive. + +If xcodebuild still exits for another reason, the next command detects the stale runner through process/liveness checks. +That command also avoids the old 15-second graceful-shutdown wait. +The remaining latency is fresh xcodebuild runner startup, not a stale transport stall. + +A `close` then `open` on a runner that still drains abandoned main-thread work now starts a fresh runner. +It does not inherit the stalled runner. +Thus, the wedge no longer survives the close/open cycle. +When retention is requested, runners that are ready and have no unanswered exchanges can remain warm. +iOS-family simulators also require an observed live listener. +Close retires unfinished startup and its recovery authority. +The retained generation owns its idle timer and listener observation. +Successful reuse ends retention. +Listener loss or idle expiry disposes that generation. +Graceful daemon shutdown stops idle retained generations before it exits. +Active ready generations retain the existing handoff policy. + +The daemon no longer models a generic "recent success" cache as a runner-health signal. +A proven healthy mutating response for the same app is now a real end-to-end liveness proof, from the HTTP listener through to the app target. +That response is recorded only after the `runnerFatal` check and only for allowlisted interactions. +Thus, a repeated series of allowlisted interactions skips the per-command `uptime` request. +Each skip is still earned again from another healthy mutation. +The earlier unconditional `uptime` before every ready-session command remains the default for non-allowlisted commands and after any of these: + +- Invalidation. +- A stale record. +- An app-bundle change. +- An absent record. + +XCTest may still be unable to examine deeply an app with a broken accessibility tree. +However, one failed snapshot no longer causes the runner to keep using a suspect cached app target. +It also no longer causes the runner to amplify the failure with a walk of every interactive element query. + +An `XCTEST_RECORDED_FAILURE` returned after a tap is treated as an ambiguous outcome at the daemon boundary. +If a usable retained snapshot exists, the daemon takes one same-presentation post-action capture. +A changed accessibility digest is reported as success with an explicit warning. +Thus, agents do not blindly repeat a tap that may already have navigated. +Unchanged, sparse, mismatched, or unavailable evidence remains the original failure. + +iOS-family Simulator and physical iOS relaunch keep the healthy XCTest process warm without carrying an app target across process identity. +The reset adds one local runner request instead of the cost of a runner restart. +Before the replacement target is reacquired, the reset clears the old process's hostile-screen capture penalty. + +Future optimization work should reduce these `uptime` preflights only after the runner exposes status that survives command-induced XCTest teardown and can prove that the session still serves new requests. diff --git a/docs/adr/0006-daemon-rpc-protocol-version.md b/docs/adr/0006-daemon-rpc-protocol-version.md index c286210963..432ae7a8f3 100644 --- a/docs/adr/0006-daemon-rpc-protocol-version.md +++ b/docs/adr/0006-daemon-rpc-protocol-version.md @@ -4,115 +4,141 @@ Accepted +## Rules at a glance + +- Before remote JSON-RPC requests, compare the daemon's `rpcProtocolVersion` from `/health`. +- Package `version` is diagnostic only and must not be used as a compatibility gate. +- Allow a missing `rpcProtocolVersion` unless a later security or protocol decision explicitly retires legacy compatibility. +- A persistent client may cache a successful probe if `/health` advertises an instance ID for the daemon endpoint and, for a proxy, for its upstream daemon. +- Send those instance IDs as conditional headers on every cached RPC. Peers without complete instance IDs retain per-command probes. +- After a stale-instance refusal, clear the cache and probe `/health` again. Retry the refused RPC once, and only if the new peer's protocol is compatible. +- A peer that advertises an instance ID must continue to honor the conditional RPC headers in future versions. +- Change `rpcProtocolVersion` only when older and newer peers cannot safely communicate over the HTTP RPC boundary for existing commands. +- Do not change `rpcProtocolVersion` for additive changes. +- Treat an absent `capabilities` entry in `daemon.json` as "cannot do this". Never infer a capability from the package version. +- A client never signals a daemon that did not accept a quiesce, except an unreachable daemon it retires deliberately. +- After `draining`, force-retire a daemon at the deadline only when it is still running, fails its liveness probes, and its command line proves it is a daemon process. + ## Context -`agent-device` can run a client on one machine and reach a daemon on another machine through the -HTTP JSON-RPC transport, including `agent-device proxy` in front of a local macOS daemon. Client and -remote host package versions can differ. Many version skews are safe because the client sends a -stable command request envelope and the daemon executes only the requested command. +`agent-device` can run a client on one machine and reach a daemon on another machine through the HTTP JSON-RPC transport. +This includes `agent-device proxy` in front of a local macOS daemon. +Client and remote host package versions can differ. +Many version differences are safe because the client sends a stable command request envelope and the daemon executes only the requested command. -Package semver is too strict for this boundary: a 0.15 client and 0.17 remote daemon can often -interoperate. At the same time, the client needs a cheap way to fail before sending command RPC when -the transport protocol itself is no longer compatible. +Package semver is too strict for this boundary. +A 0.15 client and 0.17 remote daemon can often interoperate. +However, the client needs a low-cost way to fail before it sends command RPC if the transport protocol itself is no longer compatible. ## Decision -The daemon HTTP `/health` payload advertises a `rpcProtocolVersion` integer alongside service and -package version metadata. Remote clients compare this value before sending JSON-RPC requests. +The daemon HTTP `/health` payload advertises a `rpcProtocolVersion` integer beside service and package version metadata. +Remote clients compare this value before they send JSON-RPC requests. -Package `version` is diagnostic only and must not be used as a compatibility gate. Missing -`rpcProtocolVersion` is treated as a legacy remote daemon and is allowed unless a later security or -protocol decision explicitly retires legacy compatibility. +Package `version` is diagnostic only and must not be used as a compatibility gate. +A missing `rpcProtocolVersion` is treated as a legacy remote daemon. +A legacy remote daemon is allowed unless a later security or protocol decision explicitly retires legacy compatibility. -A persistent client may cache a successful probe when `/health` advertises an instance ID for -both the daemon endpoint and, if proxied, its upstream daemon. It sends those IDs as conditional -headers on every cached RPC. A daemon refuses a stale instance with a typed HTTP 409 response -before dispatch, and a proxy refuses its own stale instance before forwarding. The proxy passes -the expected upstream instance to the daemon, which makes the same pre-dispatch check. On a -mismatch the client clears the cache, probes `/health` again, and retries the refused RPC once -only when the new peer's protocol is compatible. Peers without complete instance IDs retain -per-command probes. A peer that advertises an instance ID must continue honoring the conditional -RPC headers in future versions: older clients may hold a cached identity across a restart, and -must receive a pre-dispatch refusal before retrying a mutating command. +A persistent client may cache a successful probe if `/health` advertises an instance ID for both the daemon endpoint and, if proxied, its upstream daemon. +The client sends those IDs as conditional headers on every cached RPC. +Before dispatch, a daemon refuses a stale instance with a typed HTTP 409 response. +Before it forwards a request, a proxy refuses its own stale instance. +The proxy passes the expected upstream instance to the daemon, which makes the same pre-dispatch check. +On a mismatch, the client clears the cache and probes `/health` again. +The client then retries the refused RPC once, and only if the new peer's protocol is compatible. +Peers without complete instance IDs retain per-command probes. +A peer that advertises an instance ID must continue to honor the conditional RPC headers in future versions. +The reason is that older clients may hold a cached identity across a restart. +Such clients must receive a pre-dispatch refusal before they retry a mutating command. -`rpcProtocolVersion` changes only when an older client and newer daemon, or newer client and older -daemon, cannot safely communicate over the HTTP RPC boundary for existing commands. Bump it for -breaking changes to: +`rpcProtocolVersion` changes only when an older client and newer daemon, or a newer client and older daemon, cannot safely communicate over the HTTP RPC boundary for existing commands. +Increment it for incompatible changes to: - HTTP route requirements for `/health`, `/rpc`, `/upload`, or `/artifacts/*`. - Authentication semantics required to authorize RPC, upload, or artifact requests. -- JSON-RPC envelope shape, method naming, request id handling, or command request projection. +- JSON-RPC envelope shape, method names, request id handling, or command request projection. - Response, error, artifact, upload, or progress-stream framing that existing clients parse. -- Existing command request or response contracts when the old side would misinterpret the payload - rather than fail clearly. +- Existing command request or response contracts, if the old side would misinterpret the payload instead of failing clearly. -Do not bump it for additive changes: +Do not increment it for additive changes: - New commands. -- New optional request fields or flags that older daemons can ignore or reject with a normal - command error. +- New optional request fields or flags that older daemons can ignore or reject with a normal command error. - New optional response fields that older clients can ignore. - Package version changes, refactors, or implementation-only daemon/proxy changes. -When a single command needs finer-grained compatibility in the future, prefer command-level feature -or capability metadata over bumping the whole RPC protocol, unless the shared transport contract is -also affected. +If a single command needs finer-grained compatibility in the future, prefer command-level feature or capability metadata to an increment of the whole RPC protocol. +This preference does not apply if the change also affects the shared transport contract. ## Alternatives Considered -- Gate by package version: simple, but rejects compatible version skew and makes proxy usage brittle - across frequent releases. -- Do not check compatibility: maximizes compatibility but fails later, after the client has sent an - RPC payload that a remote daemon may parse incorrectly. -- Full schema negotiation: more precise, but too much machinery for the current JSON-RPC boundary. +- **Gate by package version.** This is simple, but it rejects compatible version differences. + It also makes proxy use brittle across frequent releases. +- **Do not check compatibility.** This maximizes compatibility. + However, it fails later, after the client has sent an RPC payload that a remote daemon may parse incorrectly. +- **Negotiate full schemas.** This is more precise, but it is too much machinery for the current JSON-RPC boundary. ## Consequences -Protocol-breaking changes must update `DAEMON_RPC_PROTOCOL_VERSION`, tests that assert `/health` -metadata, and at least one remote-client regression test that proves mismatched protocols fail before -command RPC. +Protocol-breaking changes must update: + +- `DAEMON_RPC_PROTOCOL_VERSION`. +- Tests that assert `/health` metadata. +- At least one remote-client regression test that proves mismatched protocols fail before command RPC. -The bump rules above are enforced rather than remembered (#1432). `test/wire-compat/surface.ts` -declares the wire surface grouped by the bullets in this decision, and `test/wire-compat/ledger.json` -records what each declaration hashes to and at which protocol version. The unit lane fails when a -declaration's shape moves without the ledger following, and `pnpm check:daemon-wire-compat` compares -the ledger against the last released tag: drift since that release must carry either a bump or a -`compatibleChanges` acknowledgment invoking the additive list above. Where a bullet is only partly -digestible — the `/health` and `/rpc` path literals inside `http-server.ts` — the manifest records -the gap and its reason instead of implying coverage. See `test/wire-compat/README.md`. +Checks enforce the increment rules above, so nobody has to remember them (#1432). +`test/wire-compat/surface.ts` declares the wire surface, grouped by the bullets in this decision. +`test/wire-compat/ledger.json` records the hash of each declaration and the protocol version of that hash. +The unit lane fails if the shape of a declaration changes and the ledger does not change with it. +`pnpm check:daemon-wire-compat` compares the ledger with the last released tag. +Drift since that release must carry either a protocol increment or a `compatibleChanges` acknowledgment that invokes the additive list above. +One bullet is only partly digestible: the `/health` and `/rpc` path literals inside `http-server.ts`. +For that gap, the manifest records the gap and its reason instead of implying coverage. +See `test/wire-compat/README.md`. -Legacy remote daemons without `rpcProtocolVersion` remain reachable. This keeps the first release of -the proxy compatible with older HTTP daemons, but it means absence of the marker is not proof of -compatibility. +Legacy remote daemons without `rpcProtocolVersion` remain reachable. +This keeps the first release of the proxy compatible with older HTTP daemons. +However, the absence of the marker is not proof of compatibility. ## Daemon capabilities and quiesce -A daemon on the local state directory publishes `capabilities` in `daemon.json`. This is the -command-level metadata the decision above prefers to a protocol bump. A daemon that predates a -capability omits it, so a client treats absence as "cannot do this" and never infers it from the -package version. +A daemon on the local state directory publishes `capabilities` in `daemon.json`. +This is the command-level metadata that the decision above prefers over a protocol version bump. +A daemon that predates a capability omits it. +A client treats an absent capability as "cannot do this". +A client never infers a capability from the package version. `quiesce` is how a client replaces an incompatible daemon without destroying other clients' work. The client sends `daemon_quiesce` with the daemon token and the pid and start time recorded in -`daemon.json`. The daemon checks the token and that it is that instance, refuses a request that -reached it as a remote-origin one, then closes admission and counts what it holds in a single -synchronous step: requests in flight, sessions, repair sessions and live leases, and whether the -daemon is still starting up (`startingUp`), because before that it has no drain path. Any of them -refuses with `daemon-busy` and reopens admission in that same step. Otherwise it answers -`draining`, keeps admission closed, and shuts down. The request is answered before dispatch, so -it joins neither the in-flight count nor the command registry. +`daemon.json`. +The daemon checks the token and checks that it is that instance. +The daemon refuses a request that reached it as a remote-origin request. +Then the daemon closes admission and counts what it holds in one synchronous step: + +- Requests in flight. +- Sessions. +- Repair sessions and live leases. +- Whether the daemon is still starting up (`startingUp`). Before startup completes, the daemon has no drain path. + +If any of them is present, the daemon refuses with `daemon-busy` and reopens admission in that same step. +Otherwise, the daemon answers `draining`, keeps admission closed, and shuts down. +The daemon answers the request before dispatch, so it joins neither the in-flight count nor the command registry. The daemon accepts a quiesce only when it can prove its own start time and the request names both -the pid and that start time; a daemon that cannot is never addressed. - -After `draining` the client signals nothing and waits, up to a bounded deadline, for the daemon to -exit by itself and release its registration: the daemon runs its own shutdown, which a signal would -cut short. Only a daemon still running at the deadline that also fails its liveness probes is -force-retired, and only when its command line proves it is a daemon process. A daemon that still -answers is left alone and the call fails with `daemon-drain-timeout`. A client never signals a -daemon that did not accept a quiesce, except an unreachable one it retires deliberately. - -The census counts what the contract names and nothing else. A resumable upload still streaming and -a host-administration human-control hold that names no lease are outside it, the same boundary -idle reap draws, so a quiesce can end them. A daemon started with an HTTP auth hook marks every -HTTP request remote-origin and so refuses a quiesce over HTTP; the socket transport is unaffected. +the pid and that start time. A daemon that cannot prove this is never addressed. + +After `draining`, the client sends no signal. The client waits, up to a bounded deadline, for the +daemon to exit by itself and release its registration. The daemon runs its own shutdown. A signal +would cut that shutdown short. +A daemon is force-retired only when it is still running at the deadline, fails its liveness probes, +and its command line proves it is a daemon process. +A daemon that still answers is left alone. The call fails with `daemon-drain-timeout`. +A client never signals a daemon that did not accept a quiesce. The exception is an unreachable +daemon that the client retires deliberately. + +The census counts what the contract names and nothing else. +A resumable upload that is still streaming is outside the census. A host-administration +human-control hold that names no lease is also outside it. This is the same boundary that the idle +reap draws. So a quiesce can end both. +A daemon started with an HTTP auth hook marks every HTTP request as remote-origin. Such a daemon +refuses a quiesce over HTTP. The socket transport is unaffected. diff --git a/docs/adr/0007-remote-device-leases.md b/docs/adr/0007-remote-device-leases.md index b69a1f181e..b829062b1f 100644 --- a/docs/adr/0007-remote-device-leases.md +++ b/docs/adr/0007-remote-device-leases.md @@ -4,218 +4,288 @@ Accepted +## Rules at a glance + +- Allocate device leases lazily, when the device, backend, and provider are known. +- Run lease admission, heartbeat, stored session lease refresh, and request execution under the same daemon request lock. +- Do not mutate lease ownership before the lock. Scope resolution may happen before the lock. +- Generated connection profiles must strip daemon and Metro bearer tokens. +- A session cannot outlive its lease. Lease-end teardown clears the session's device claim even when the device no longer answers. +- Report each lease that a request ended once, at response level. A client never sends a release after such a request. +- Admitted work that is still wanted preserves its lease. If the client hung up, the request neither defers expiry past that cancellation nor renews the lease. +- Before admission, send lease heartbeats for as long as a client phase runs, for example an artifact upload. Renew under the window that the caller named, or else under the lease's existing window. +- Before a pending human-control activation reports active, it fences new mutations and drains the mutations already admitted. +- Only host administrators allocate `macos-app` leases. Tenant `lease_release` is refused. +- Confine each request admitted under a `macos-app` lease to the leased app. + ## Context -Remote daemon users need a clear ownership boundary before commands reach a -platform runner or helper. Shared proxy and hosted providers need ownership to -include the selected device and connection provider, not only tenant/run. +Remote daemon users need a clear ownership boundary before commands reach a platform runner or helper. +Shared proxies and hosted providers need ownership to include the selected device and connection provider, not only tenant/run. -Runner and helper processes already have backend-specific mutual exclusion. That -guard protects platform tooling, not remote client ownership, so surfacing those -errors directly makes device contention harder to recover from. +Runner and helper processes already have backend-specific mutual exclusion. +That guard protects platform tooling, not remote client ownership. +Thus, direct exposure of those errors makes device contention harder to recover from. ## Decision -A remote device lease is logical ownership of one selected device by one -remote client for a connection provider such as `proxy`, cloud, or `limrun`. - -`connect` establishes connection profile and client identity. Lease allocation -is lazy and happens when a device, backend, and provider are known. - -A runner/process lease is a backend helper guard and is not a user/client -ownership boundary. It stays below daemon device leases and should not be -weakened or replaced by them. - -`open` is the natural point to acquire a device lease because target resolution -and session creation meet there. Commands after `open` must refresh the lease; -no activity for five minutes should make the device available again. - -Lease admission, heartbeat, stored session lease refresh, and request execution -must run under the same daemon request lock. Scope resolution may happen before -the lock, but lease ownership mutation must not. - -Generated connection profiles are non-secret. They may persist routing and -lease metadata, but must strip daemon and Metro bearer tokens. Tokens are -supplied in-memory for the current command or through environment/CLI token -paths. - -The proxy process is expected to be long-lived and self-serve. Recovery from a -stale or expired device lease should not require restarting the proxy. - -A caller that owns a lease's lifetime, allocating it and releasing it itself, -can allocate with `retainOnClose`. Session `close` then leaves that lease and -its provider device in place, and only `leases.release`, expiry, or daemon -shutdown ends it. The default is unchanged so the CLI's proxy sharing still -frees devices on `close`. The allocated lease reports `retainOnClose: true` -only when the daemon honored it, and only the lease's own client can turn it -on for a lease the run already holds. An unexpired `retainOnClose` lease keeps -an idle daemon alive; other leases, including human-control holds, do not. A -retained lease the caller stops heartbeating expires after its `ttlMs`, and idle -reap then shuts the daemon down within one more idle window. - -A session cannot outlive the lease it was opened under. When the lease is -released (tenant `lease_release` or host `DELETE /admin/leases/`) or -expires, the daemon tears the session down under its execution locks and clears -its device claim, even when the device no longer answers. The daemon sweeps for -expired leases every second, so expiry does not wait for a later request; -in-flight work and human-control holds still keep a lease from expiring. Once -daemon shutdown begins, no new lease-end teardown starts, and shutdown waits for -any that already started before it tears sessions down itself. - -The daemon ends a session's lease when a request closes that session, unless the lease was -allocated with `retainOnClose`. The reply to that request reports each lease it ended once, at -response level, as `endedLeases: [{ leaseId, outcome: 'released' }]`: in `data` on success, in -`error.details` on failure, including a close that failed after the release. This holds for every -request that can close a session: `close`, a `batch` close step, a `replay` or `test` script's -`close`, and the session cleanup `test` runs after each script. Nested requests record on the -outermost request's ledger, so a step reply never repeats it. A close that finds its lease already -inactive reports it as ended. A client never sends a release after such a request. A client that -stores the lease id forgets it when the reply names it, and a gateway that relays the reply reads -the lease end from it. Close and lease-end teardown take the same execution locks, so whichever -runs second finds the session gone: a close after a lease end has no session and ends no lease, and -a client that still stores that lease id finds it inactive on its next heartbeat and allocates a -new lease under a fresh id. +A remote device lease is logical ownership of one selected device by one remote client for a connection provider such as `proxy`, cloud, or `limrun`. + +`connect` establishes the connection profile and client identity. +Lease allocation is lazy. +It happens when a device, backend, and provider are known. + +A runner/process lease is a backend helper guard. +It is not a user/client ownership boundary. +It stays below daemon device leases and should not be weakened or replaced by them. + +`open` is the natural point to acquire a device lease because target resolution and session creation meet there. +Commands after `open` must refresh the lease. +After five minutes without activity, the device should become available again. + +These operations must run under the same daemon request lock: + +- Lease admission. +- Heartbeat. +- Stored session lease refresh. +- Request execution. + +Scope resolution may happen before the lock. +Lease ownership mutation must not happen before the lock. + +Generated connection profiles are non-secret. +They may persist routing and lease metadata, but must strip daemon and Metro bearer tokens. +Tokens are supplied in memory for the current command or through environment/CLI token paths. + +The proxy process is expected to be long-lived and self-serve. +Recovery from a stale or expired device lease should not require a proxy restart. + +A caller that owns the lifetime of a lease, and allocates and releases it itself, can allocate with `retainOnClose`. +Session `close` then leaves that lease and its provider device in place. +Only `leases.release`, expiry, or daemon shutdown ends it. +The default is unchanged, so the CLI's proxy sharing still frees devices on `close`. +The allocated lease reports `retainOnClose: true` only if the daemon honored it. +Only the lease's own client can turn it on for a lease that the run already holds. +An unexpired `retainOnClose` lease keeps an idle daemon alive. +Other leases, including human-control holds, do not. +If the caller stops heartbeats for a retained lease, that lease expires after its `ttlMs`. +Idle reap then shuts the daemon down within one more idle window. + +A session cannot outlive the lease under which it was opened. +When the lease is released or expires, the daemon tears the session down under its execution locks and clears its device claim. +This applies even when the device no longer answers. +A release is a tenant `lease_release` or a host `DELETE /admin/leases/`. +The daemon sweeps for expired leases every second. +Thus, expiry does not wait for a later request. +In-flight work and human-control holds still keep a lease from expiring. +After daemon shutdown begins, no new lease-end teardown starts. +Shutdown waits for any lease-end teardown that already started before it tears sessions down itself. + +When a request closes a session, the daemon ends the lease of that session, unless the lease was allocated with `retainOnClose`. +The reply to that request reports each lease that it ended once, at response level, as `endedLeases: [{ leaseId, outcome: 'released' }]`. +On success, the report is in `data`. +On failure, the report is in `error.details`, including for a close that failed after the release. +This applies to every request that can close a session: + +- `close`. +- A `batch` close step. +- The `close` of a `replay` or `test` script. +- The session cleanup that `test` runs after each script. + +Nested requests record on the ledger of the outermost request, so a step reply never repeats the report. +If a close finds its lease already inactive, it reports that lease as ended. +A client never sends a release after such a request. +If a client stores the lease id, it forgets that id when the reply names it. +A gateway that relays the reply reads the lease end from it. + +Close and lease-end teardown take the same execution locks. +Thus, whichever runs second finds the session gone. +A close after a lease end has no session and ends no lease. +If a client still stores that lease id, its next heartbeat finds the lease inactive. +The client then allocates a new lease under a fresh id. ## Consequences -Device contention can fail before platform execution with an explicit -device-lease error that includes the backend, provider, selected device key, and -owning lease expiry. +Device contention can fail before platform execution with an explicit device-lease error. +That error includes the backend, provider, selected device key, and owning lease expiry. -Backend-only leases remain valid for older remote clients, while provider-aware -clients get device-level contention and clearer recovery. +Backend-only leases remain valid for older remote clients. +Provider-aware clients get device-level contention and clearer recovery. ## Admitted request work -A lease renews when a request is admitted and never again while that request runs, so an admitted -command slower than the inactivity TTL used to expire the lease that was paying for its own device -and tear the session down underneath the client still waiting for its result. Admitted work -therefore preserves its lease the way a human-control hold does: while the request is still wanted -it defers expiry, and finishing while still wanted renews the lease for its existing inactivity TTL -from the moment the work ended. A request whose client hung up preserves nothing — it neither defers -expiry past that cancellation nor renews the lease when it finally lands — so a handler that ignores -its cancellation cannot hold a rented device open. - -Which leases this reaches depends on the inactivity TTL the client asked for: the daemon default is -one minute, while a cloud WebDriver connection profile asks for ten. A single command that runs -longer than its own lease is therefore ordinary on the default and only reachable through a profile -on the longer one. +A lease renews when a request is admitted, and never again while that request runs. +Thus, an admitted command slower than the inactivity TTL used to expire the lease that paid for its own device. +The daemon then tore the session down while the client still waited for its result. +Admitted work therefore preserves its lease in the same way as a human-control hold does. +While the request is still wanted, it defers expiry. +If the request finishes while it is still wanted, it renews the lease for its existing inactivity TTL from the moment the work ended. +A request whose client hung up preserves nothing. +It neither defers expiry past that cancellation nor renews the lease when the request finally finishes. +Thus, a handler that ignores its cancellation cannot hold a rented device open. + +The inactivity TTL that the client asked for determines which leases this preservation reaches. +The daemon default is one minute. +A cloud WebDriver connection profile asks for ten minutes. +Thus, a single command that runs longer than its own lease is ordinary on the default. +On the longer window, such a command is only reachable through a profile. ## Provider allocation -Allocation is admitted work too. The registry records a lease before a hosted provider creates the -session behind it, and creating that session can take longer than the lease's inactivity TTL, so a -lease timed from its record was already expired, or nearly so, when its client first received it, and -the paid session it pointed at was orphaned. The allocation therefore holds a work pass while the -provider allocates: the lease cannot expire underneath it, and a successful allocation still wanted by -its requester starts the inactivity TTL from the moment allocation completed. A requester that hung up -preserves nothing, and its allocation is released as before. +Allocation is admitted work too. +The registry records a lease before a hosted provider creates the session behind it. +Creation of that session can take longer than the inactivity TTL of the lease. +Thus, a lease timed from its record was already expired, or nearly expired, when its client first received it. +The paid session that the lease pointed at was orphaned. +The allocation therefore holds a work pass while the provider allocates. +The lease cannot expire during that allocation. +If a successful allocation is still wanted by its requester, the inactivity TTL starts from the moment allocation completed. +A requester that hung up preserves nothing, and its allocation is released as before. ## Client-side work that precedes admission -Protecting admitted work covers nothing that happens before a request is admitted. Installing an -artifact uploads it from the caller while the request that will consume it has not been admitted yet, -so an upload slower than the lease's inactivity window expired the lease paying for the device the -bytes were going to (#2946). The caller therefore beats the lease over the ordinary transport for as -long as that phase runs, naming the lease scope exactly as the command named it and no payload of its -own. An install names no window, so its beats renew for the window the lease already carries; a -caller that did name one keeps renewing on it. Resolving an absent window to the registry default -instead — which is what a heartbeat used to do — quietly shortened every lease allocated above that -default, which is the other half of why the upload could not survive. Request admission had its own -copy of that mistake: it named a proxy-specific default for every admitted request, so a lease -allocated longer than the default lost its window on the next command. Admission renews on the lease's -window too; the window a lease carries is the one its client named when it allocated. - -The first beat is fired when the phase starts, not one cadence in, because a beat is what proves the -lease the upload is spending time on is still alive — a lease shorter than any fixed cadence would -otherwise lapse before anything renewed it. A lease already gone is caught early, but not before the -phase begins: hashing, the preflight, and the start of the stream can all run while the first beat is -still outstanding, so what the first beat buys is that the loss is learned during the upload rather -than after it. Each beat answers with the window it just renewed, and the loop arms its successor when -the beat starts rather than when it settles, so a beat that never answers is abandoned on schedule -instead of taking the schedule with it. A phase that settles does not wait on a beat still in flight. - -Cadence and budget are deliberately two numbers. A beat is armed every third of the window, but it is -allowed the whole window to answer in. Budgeting a beat at its cadence instead would give a heartbeat -that needs more than a third of a window for its round trip — a tunneled proxy, an uplink the upload -itself is saturating — no way to answer at all, and every beat would re-time-out on exactly the slow -links the beat exists to protect. The window, not the cadence, is the interval in which being alive -still matters, so it is also the longest a beat's answer is worth waiting for. - -That ceiling is the shorter of the window and the heartbeat's own request policy, currently ninety -seconds. A ten-minute lease therefore gives a beat ninety seconds rather than the full window, which -costs nothing the beat exists to buy: the daemon renews the lease when it handles the request, before -it answers, so a round trip abandoned at the cap has still extended the lease if the request ever -landed — and if it never landed, a transport that has stopped carrying traffic is not served by holding -the socket open for the rest of a window measured in minutes. What the cap refuses is exactly that -idle hold. It binds only on windows longer than the policy, and on those the beat already has the -longest answer deadline any lease gets; shorter windows are bounded by their own window, as above. - -A beat is a fresh request each time, never the protected request rewritten: a request identity is -what a timed-out beat is canceled under, and beats must not inherit each other's cancellation. A beat -that finds the lease gone, or finds that this request can never renew it — its scope is missing or -belongs to another lease, or the daemon rejects the scope and window the beat itself was built with, -which no successor will ask differently — ends the phase with that error and cancels the upload rather -than finishing bytes against a device nobody owns or a lease that will stop renewing. A beat that fails -for any other reason is reported and survived, because a later beat covers one lost request — including -one that was abandoned at its budget rather than answered, which is what makes that promise hold for a -stalled round trip and not only for one that fails fast. +The protection of admitted work covers nothing that happens before a request is admitted. +An artifact install uploads the artifact from the caller before the request that will consume it is admitted. +Thus, an upload slower than the inactivity window of the lease expired the lease that paid for the target device of the bytes (#2946). +The caller therefore sends lease heartbeats over the ordinary transport for as long as that phase runs. +Each heartbeat names the lease scope exactly as the command named it, and carries no payload of its own. +An install names no window, so its heartbeats renew for the window that the lease already carries. +A caller that did name a window continues to renew on that window. + +A heartbeat used to resolve an absent window to the registry default instead. +That resolution shortened, without notice, every lease allocated above that default. +That was the other half of the reason that the upload could not survive. +Request admission had its own copy of that mistake. +It named a proxy-specific default for every admitted request. +Thus, a lease allocated longer than the default lost its window on the next command. +Admission also renews on the window of the lease. +The window that a lease carries is the one that its client named when it allocated. + +The first heartbeat is sent when the phase starts, not one cadence later. +The reason is that a heartbeat proves that the lease on which the upload spends time is still alive. +Otherwise, a lease shorter than any fixed cadence would lapse before anything renewed it. +A lease that is already gone is caught early, but not before the phase begins. +Hashing, the preflight, and the start of the stream can all run while the first heartbeat is still outstanding. +Thus, the first heartbeat makes sure that the loss is learned during the upload instead of after it. +Each heartbeat answers with the window that it just renewed. +The loop arms the next heartbeat when the current heartbeat starts, not when it settles. +Thus, a heartbeat that never answers is abandoned on schedule and does not stop the schedule. +A phase that settles does not wait for a heartbeat still in flight. + +Cadence and budget are deliberately two numbers. +A heartbeat is armed every third of the window, but it is allowed the whole window to answer. +If the budget were equal to the cadence, a heartbeat that needs more than a third of a window for its round trip could never answer. +Examples are a tunneled proxy and an uplink that the upload itself saturates. +Every heartbeat would then time out again on exactly the slow links that the heartbeat exists to protect. +The window, not the cadence, is the interval in which liveness still matters. +Thus, the window is also the longest time that an answer to a heartbeat is worth a wait. + +That ceiling is the shorter of the window and the heartbeat's own request policy, currently ninety seconds. +Thus, a ten-minute lease gives a heartbeat ninety seconds instead of the full window. +This limit costs nothing that the heartbeat exists to provide. +The daemon renews the lease when it handles the request, before it answers. +Thus, a round trip abandoned at the cap has still extended the lease if the request ever arrived. +If the request never arrived, the transport has stopped carrying traffic. +A socket held open for the rest of a window measured in minutes does not help that transport. +The cap refuses exactly that idle hold. +The cap binds only on windows longer than the policy. +On those windows, the heartbeat already has the longest answer deadline that any lease gets. +Shorter windows are bounded by their own window, as above. + +A heartbeat is a fresh request each time, never the protected request rewritten. +The reason is that a timed-out heartbeat is canceled under its request identity, and heartbeats must not inherit each other's cancellation. +In these cases, a heartbeat ends the phase with that error and cancels the upload: + +- The heartbeat finds that the lease is gone. +- The heartbeat finds that this request can never renew the lease. + This is so if the scope is missing or belongs to another lease. + This is also so if the daemon rejects the scope and window that the heartbeat itself was built with, which no later heartbeat will ask differently. + +Thus, the upload does not finish bytes against a device that nobody owns or a lease that will stop renewing. +A heartbeat that fails for any other reason is reported, and the phase continues. +The reason is that a later heartbeat covers one lost request. +This includes a request abandoned at its budget instead of answered. +Thus, that promise holds for a stalled round trip, not only for a round trip that fails fast. ## Human control -Human-control holds coexist with an open remote session. They belong to `LeaseRegistry` and use -the same backend/provider/device contention key as device-aware leases. Hold heartbeat, expiry, -lease preservation, and release refresh are one registry-owned lifecycle. Releasing or expiring the -last hold gives the lease its existing inactivity TTL again; expiry uses the hold's expiry instant. - -Tenant hold operations are ordinary daemon RPCs admitted through `request-admission.ts`. Their -device comes only from the admitted lease. Host administration is a distinct loopback capability -authenticated with the daemon token, never a tenant credential; tenants cannot modify host holds. - -Mutation admission derives from existing recording effects, observation-class inventory, and -observability semantics. Takeover and lease heartbeats are exempt from the mutation fence, not from -their ownership checks. Unknown effects are treated as mutations. A pending activation fences new -mutations and drains those already admitted before reporting active; advisory execution locks alone -do not establish this guarantee for fresh sessions. - -Activation follows the calling RPC or host HTTP request's cancellation signal. A disconnect while -draining removes only that request's pending hold, leaving successor and unrelated holds intact; -canceling activation does not cancel the mutations being drained. Completed holds use their TTL or -explicit release lifecycle. - -Holds and ordinary proxy leases are in-memory and do not survive daemon restart. Controllers must -reconnect and re-establish them; no persisted hold store is used. Local takeover is deferred: a -future host-global human-control fence must coexist with the local session's device claim, not -acquire it exclusively. +Human-control holds coexist with an open remote session. +They belong to `LeaseRegistry`. +They use the same backend/provider/device contention key as device-aware leases. +Hold heartbeat, expiry, lease preservation, and release refresh are one registry-owned lifecycle. +Release or expiry of the last hold gives the lease its existing inactivity TTL again. +Expiry uses the hold's expiry instant. + +Tenant hold operations are ordinary daemon RPCs admitted through `request-admission.ts`. +Their device comes only from the admitted lease. +Host administration is a distinct loopback capability authenticated with the daemon token, never a tenant credential. +Tenants cannot modify host holds. + +Mutation admission derives from existing recording effects, observation-class inventory, and observability semantics. +Takeover and lease heartbeats are exempt from the mutation fence, not from their ownership checks. +Unknown effects are treated as mutations. +A pending activation fences new mutations and drains the mutations already admitted before it reports active. +Advisory execution locks alone do not establish this guarantee for fresh sessions. + +Activation follows the cancellation signal of the calling RPC or host HTTP request. +A disconnect during the drain removes only the pending hold of that request. +Successor and unrelated holds remain intact. +A canceled activation does not cancel the mutations under drain. +Completed holds use their TTL or explicit release lifecycle. + +Holds and ordinary proxy leases are in-memory and do not survive daemon restart. +Controllers must reconnect and establish them again. +No persisted hold store is used. +Local takeover is deferred. +A future host-global human-control fence must coexist with the device claim of the local session, not acquire it exclusively. ## macOS app leases -A lease on the macOS host would rent the whole desktop, so the macOS platform rents one app instead. -A `macos-app` lease's device key is a bundle id, optionally pinned to one process -(`` or `@`). The scope would be self-chosen if a tenant could name it, so -only a host administrator allocates one, over the loopback `/admin/leases` route that uses the daemon -token like host holds; tenant `lease_allocate` refuses the backend. The host picks the lease id, a -repeated PUT renews it, and a PUT naming another scope for an existing id is refused rather than -rewritten. Heartbeat, expiry, and the loss on daemon restart are those of any lease, except that a -tenant heartbeat or request cannot renew it for longer than the window of the host's last PUT, and -a tenant cannot release it: `lease_release` is refused with `MACOS_APP_LEASE_HOST_OWNED`. The lease -ends by the host's `DELETE /admin/leases/`, by expiry, by the loss on daemon restart, or -when a session it holds closes and the host set `retainOnClose` to false. - -Request admission confines every request admitted under the lease, so `batch` steps and `replay` -actions are confined when they re-enter it: an allow list of commands, the ones whose command -registry descriptor declares `appLease: 'allowed'` (later commands are refused, and of the commands -lease admission otherwise exempts only `lease_heartbeat` and `lease_release` declare it), -`open` and `close` of the leased bundle only, the `app` surface only, window-only screenshots, no -input that names a host path or launches beside the app, and an existing session that is the leased -app for every request but `open` and the `batch` envelope, so a -request naming no session cannot fall back to the host Mac. `open` requires the native app backend (ADR 0031), because XCTest -posts screen events that can land outside the app's window. A pid-pinned lease is checked against the -running process before each admitted request. A session opened under the lease holds its app, not -the Mac: it takes no host device claim, and other app-leased sessions on the same Mac do not conflict -with it, so one daemon serves several leased apps beside the host's own sessions. - -These rules bind a request that names the lease or runs in its session. A daemon policy -`leases.require` (ADR 0029) refuses requests that name no lease. The proxy token is one credential -for every client, so a host serving several clients through one proxy authenticates each client and -sets its tenant, session isolation, and lease on every request it forwards. +A lease on the macOS host would rent the whole desktop. +Thus, the macOS platform rents one app instead. +The device key of a `macos-app` lease is a bundle id, optionally pinned to one process (`` or `@`). +If a tenant could name the scope, the scope would be self-chosen. +Thus, only a host administrator allocates a `macos-app` lease, over the loopback `/admin/leases` route that uses the daemon token like host holds. +Tenant `lease_allocate` refuses the backend. +The host picks the lease id. +A repeated PUT renews the lease. +A PUT that names another scope for an existing id is refused, not rewritten. + +Heartbeat, expiry, and the loss on daemon restart are those of any lease, with two exceptions: + +- A tenant heartbeat or request cannot renew the lease for longer than the window of the host's last PUT. +- A tenant cannot release the lease. `lease_release` is refused with `MACOS_APP_LEASE_HOST_OWNED`. + +The lease ends through one of these: + +- The host's `DELETE /admin/leases/`. +- Expiry. +- The loss on daemon restart. +- The close of a session that the lease holds, if the host set `retainOnClose` to false. + +Request admission confines every request admitted under the lease. +Thus, `batch` steps and `replay` actions are confined when they enter admission again. +The confinement has these parts: + +- An allow list of commands: the commands whose command registry descriptor declares `appLease: 'allowed'`. + A command without that declaration is refused, including a command that is added later. + Of the commands that lease admission otherwise exempts, only `lease_heartbeat` and `lease_release` declare it. +- `open` and `close` of the leased bundle only. +- The `app` surface only. +- Window-only screenshots. +- No input that names a host path or launches beside the app. +- For every request except `open` and the `batch` envelope, an existing session that is the leased app. + +Thus, a request that names no session cannot fall back to the host Mac. +`open` requires the native app backend (ADR 0031), because XCTest posts screen events that can land outside the app's window. +Before each admitted request, a pid-pinned lease is checked against the running process. +A session opened under the lease holds its app, not the Mac. +It takes no host device claim. +Other app-leased sessions on the same Mac do not conflict with it. +Thus, one daemon serves several leased apps beside the host's own sessions. + +These rules bind a request that names the lease or runs in its session. +A daemon policy `leases.require` (ADR 0029) refuses requests that name no lease. +The proxy token is one credential for every client. +Thus, a host that serves several clients through one proxy authenticates each client. +The host sets the tenant, session isolation, and lease of that client on every request that it forwards. diff --git a/docs/adr/0008-command-descriptor-registry.md b/docs/adr/0008-command-descriptor-registry.md index 25bfeeec96..337fec37e5 100644 --- a/docs/adr/0008-command-descriptor-registry.md +++ b/docs/adr/0008-command-descriptor-registry.md @@ -4,104 +4,131 @@ Accepted -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** `CommandDescriptor` remains -> the command declaration root. Device-command capability buckets evolve into typed required and -> preferred runtime use joined with leaf/device/provider-specific runtime facts; inventory commands -> declare inventory use instead of fabricating a device binding. The descriptor's other derived -> projections remain accepted. +Completed process history is in git history. + +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** `CommandDescriptor` remains the root for command declarations. +> Device-command capability buckets evolve into typed required and preferred runtime use joined with leaf/device/provider-specific runtime facts. +> Inventory commands declare inventory use instead of a fabricated device binding. +> The descriptor's other derived projections remain accepted. > -> **Parity gate retired (2026-09).** The migration rule below — assert each derived table -> byte-for-byte against its hand-authored table before deleting the hand table — was followed, and -> completing it removed the gate's second operand. Each hand table went away as its projection was -> inverted out: #907 daemon routes, #908 capability matrix, #909 batch allowlist, #1084 -> daemon-client timeout lists, #1137 MCP exposure list. No parity test exists. Do not add one or -> look for one; the totality invariants are now carried by compile-time guards -> (`CommandOwnerFileClaimsAreComplete`) and by literal pin-set tests that compare a name set rather -> than two tables. +> **Parity gate retired (2026-09).** The completed migration deleted every hand-authored table, so a parity test has no second table to compare. +> No byte-for-byte table parity test exists. +> Do not add or search for one. +> Compile-time guards (`CommandOwnerFileClaimsAreComplete`) and literal pin-set tests now enforce totality. +> Those tests compare name sets instead of two tables. + +## Rules at a glance + +- Compose one `CommandDescriptor` per command from facets owned by their domains. +- Derive every consumer table through pure projection. +- Keep narrow command traits that affect multiple projections on the descriptor. +- Keep the cross-process client `invoke` seam and the in-daemon `execute` seam distinct. Never collapse the process boundary. +- Preserve all four invariants from ADR 0003. +- The descriptor module must own the `Input`/`Result` types. The client must be derived as a view type, and a lint boundary enforces this. ## Context -A command's identity is restated, by hand, across roughly ten tables that must stay aligned by -convention: `PUBLIC_COMMANDS` (`src/command-catalog.ts`), the per-command metadata and family facets -(`src/commands/**`), the capability matrix (`src/core/capabilities.ts`), the daemon command registry -(`packages/daemon/src/core/daemon-command-registry.ts`, ADR 0003), the structured-batch allowlist -(`src/batch-policy.ts`), the MCP exposure sets, the Node client interface and impl (`src/client-types.ts`, -`src/client.ts`), and the generic-dispatch `switch` (`src/core/dispatch.ts`, whose `default: throw` makes a -missing or renamed command a runtime error, not a compile error). Adding one command touches ~24 files, the -argument shape is (de)serialized ~4 times, and the gesture set is retyped in three places. +Roughly ten manually maintained tables repeat command identity and require alignment by convention: + +- `PUBLIC_COMMANDS` in `src/command-catalog.ts`. +- Per-command metadata and family facets in `src/commands/**`. +- Capability matrix in `src/core/capabilities.ts`. +- Daemon command registry in `packages/daemon/src/core/daemon-command-registry.ts` (ADR 0003). +- Structured-batch allowlist in `src/batch-policy.ts`. +- MCP exposure sets. +- Node client interface and implementation in `src/client-types.ts` and `src/client.ts`. +- Generic-dispatch `switch` in `src/core/dispatch.ts`. + +The switch's `default: throw` makes a missing or renamed command a runtime error instead of a compile error. +One new command changes approximately 24 files. +Its argument shape is serialized or deserialized approximately four times. +Three places repeat the gesture set. -The codebase already proves the cure works for part of this: the `CommandFamilyFacet` -(`src/commands/family/`) derives the MCP tools, the CLI schema, and the batch writer from a single array. -It simply stops at the command-surface boundary; everything past it is hand-maintained. +`CommandFamilyFacet` in `src/commands/family/` already proves that derivation works for part of this problem. +It derives MCP tools, CLI schema, and batch writer from one array. +However, derivation stops at the command-surface boundary. +Beyond it, tables remain manually maintained. -ADR 0003 deliberately separated daemon route/policy into its own internally-owned registry with a small -predicate interface, and its 2026-06 update set four invariants that any single-declaration/derivation -model must preserve. This ADR is that model. +ADR 0003 deliberately gives daemon route/policy a separately owned internal registry with a small predicate interface. +Its 2026-06 update sets four invariants that any single-declaration/derivation model must preserve. +This ADR defines that model. ## Decision -Introduce one `CommandDescriptor` per command that **composes facets owned by their domains** and from which -every consumer table is **derived** by pure projection: - -- The descriptor composes catalog identity, command-family surface projection hooks - (`src/commands/**` remains the owner of CLI schema/reader and executable surface metadata), a - `capability` facet, a `daemon` facet (route + request-policy traits shaped by ADR 0003), platform - dispatch membership, response data transforms, and typed-result hooks. -- Narrow command traits that affect multiple projections, such as interaction post-action observation - (`--settle` / `--verify`), live on the descriptor so CLI flags, command metadata, timeout policy, and - tests derive from one fact instead of repeating command-name lists. -- The public catalog, capability matrix, daemon command registry, batch allowlist, MCP tool list, CLI - schema exposure, and descriptor-backed Node client surface become pure projections of the descriptor set. The - `src/core/dispatch.ts` `switch` is replaced by a total map keyed on the command-name union, so a missing - handler is a compile error. -- The cross-process `invoke` (client) and in-daemon `execute` seams stay distinct; the process boundary is - never collapsed. - -This **composes with**, and is bound by, ADR 0003's four invariants: daemon-owned declaration (never inlined -into the public surface), the predicate interface unchanged, no leakage of daemon-only traits into public -projections, and one declaration per concern enforced by the type system. +Introduce one `CommandDescriptor` per command. +It **composes facets owned by their domains**. +Every consumer table is **derived** through pure projection: + +- The descriptor composes catalog identity, command-family surface projection hooks, a `capability` facet, a `daemon` facet, platform dispatch membership, response data transforms, and typed-result hooks. + `src/commands/**` remains the owner of the CLI schema/reader and executable surface metadata. + ADR 0003 shapes the route and request-policy traits of the `daemon` facet. +- Narrow command traits that affect multiple projections belong to the descriptor. + Interaction post-action observation (`--settle` / `--verify`) is one example. + Thus, CLI flags, command metadata, timeout policy, and tests derive from one fact instead of repeated command-name lists. +- These surfaces become pure projections of the descriptor set: + - Public catalog. + - Capability matrix. + - Daemon command registry. + - Batch allowlist. + - MCP tool list. + - CLI schema exposure. + - Descriptor-backed Node client surface. + A total map keyed on the command-name union replaces the `src/core/dispatch.ts` `switch`. + A missing handler therefore causes a compile error. +- The cross-process client `invoke` seam and the in-daemon `execute` seam remain distinct. + The process boundary is never collapsed. + +This design composes with ADR 0003's four invariants and is bound by them: + +1. Daemon-owned declaration, never inline in the public surface. +2. Unchanged predicate interface. +3. No daemon-only traits in public projections. +4. One declaration per concern, enforced by types. ## Alternatives Considered -- Keep the hand-synced tables: no migration risk, but it is the status quo this ADR exists to remove — - ~24-file cost per command and drift kept in check only by convention and tests. -- A single flat public descriptor with daemon fields inlined: re-contaminates the public command surface - with daemon-only policy, which is exactly what ADR 0003 (and its update) forbid. -- Build-time code generation: a real option, but runtime derivation with `as const satisfies` keeps the - source of truth in type-checked TypeScript with no separate build step or generated artifacts to review. +- **Keep manually synchronized tables.** This avoids migration risk. + However, it is the status quo that this ADR exists to remove. + Each command costs approximately 24 changed files, with alignment enforced only by convention and tests. +- **Use one flat public descriptor with inline daemon fields.** This puts daemon-only policy back into the public command surface. + ADR 0003 and its update explicitly forbid this ownership combination. +- **Generate code at build time.** This is a valid option. + However, runtime derivation with `as const satisfies` retains the source of truth in type-checked TypeScript. + It requires no separate build step or generated artifacts for review. ## Consequences -Adding a plain command touches ~1–2 files; per-platform behavior remains N implementations behind the -descriptor's `execute`. The descriptor is the prerequisite for typed per-command results (ADR 0010, which -deletes the `src/client-types.ts` mirror) and supplies the capability facet the platform-plugin work -(ADR 0009) hooks into. - -Migration is **strangler-fig and sequential** — never a big-bang: - -1. Introduce the `commandRegistry` as the root and **invert the import graph** so `command-catalog`, - `capabilities`, `daemon-command-registry`, and `batch-policy` become leaves that derive from it. -2. Promote each family's facet to a full `CommandDescriptor`, family by family. -3. Replace the dispatch `switch` with the registry-driven total map, arm by arm. - -Each derived table must be asserted **byte-for-byte equivalent** to the hand-authored table by a parity test -**before** the hand table is deleted. The principal risk is the import-cycle inversion: `command-catalog.ts` -has ~95 importers and the family facet currently imports `AgentDeviceClient`, so the descriptor module must -own the `Input`/`Result` types and the client must be derived as a view type, enforced by a lint boundary. -As of 2026-07, the descriptor registry is live for command identity (`PUBLIC_COMMANDS`, -`INTERNAL_COMMANDS`, and local CLI names), daemon registry traits, capability matrix, -structured-batch allowlist, daemon-client timeout policy, MCP exposure list, -capability-checked CLI command list, post-action observation traits, and the platform dispatch -command set (including the remaining dispatch-only alias `read`). Command-family -surface metadata still lives under `src/commands/**`, where the CLI grammar and client-backed -executors already live, but it is coherence-guarded against the descriptor CLI catalog so new -surface names cannot drift from the descriptor root. - -The remaining deferred work is the public Node-client typed-result narrowing tracked outside this -registry migration in [#1153](https://github.com/callstack/agent-device/issues/1153): commands whose -daemon responses still expose dynamic payloads or whose closed result contracts live in runtime -modules must move those contracts before their client methods can stop returning -`CommandRequestResult`. - -This ADR owns the decision and its constraints; the roadmap that prototyped it has been retired, with -the delivered end-state enforced by the descriptor projections below. +A plain new command changes approximately 1–2 files. +Per-platform behavior remains N implementations behind the descriptor's `execute`. +The descriptor is the prerequisite for typed per-command results in ADR 0010, which deletes the `src/client-types.ts` mirror. +It also supplies the capability facet used by the platform-plugin work in ADR 0009. + +During the migration, the import-cycle inversion was the principal risk. +At that time, `command-catalog.ts` had approximately 95 importers. +The family facet imports `AgentDeviceClient`. +Thus, the descriptor module must own the `Input`/`Result` types. +The client must be derived as a view type, and a lint boundary enforces this. + +As of 2026-07, the descriptor registry is live for these surfaces: + +- Command identity: `PUBLIC_COMMANDS`, `INTERNAL_COMMANDS`, and local CLI names. +- Daemon registry traits. +- Capability matrix. +- Structured-batch allowlist. +- Daemon-client timeout policy. +- MCP exposure list. +- Capability-checked CLI command list. +- Post-action observation traits. +- Platform dispatch command set, including the remaining dispatch-only alias `read`. + +Command-family surface metadata remains under `src/commands/**`, beside CLI grammar and client-backed executors. +Coherence checks compare it with the descriptor CLI catalog. +Thus, new surface names cannot differ from the descriptor root. + +The remaining deferred work is the public Node-client typed-result narrowing. +[#1153](https://github.com/callstack/agent-device/issues/1153) tracks this work outside this registry migration. +Some commands still expose dynamic daemon response payloads or retain closed result contracts in runtime modules. +Before their client methods can stop returning `CommandRequestResult`, those contracts must move. + +This ADR owns the decision and its constraints. +The descriptor projections enforce the delivered end state. diff --git a/docs/adr/0009-apple-platform-consolidation.md b/docs/adr/0009-apple-platform-consolidation.md index a77ad70c29..02c538c6d0 100644 --- a/docs/adr/0009-apple-platform-consolidation.md +++ b/docs/adr/0009-apple-platform-consolidation.md @@ -4,84 +4,125 @@ Accepted -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** The Apple family and explicit -> `AppleOS` leaf axis remain accepted. The shallow `PlatformPlugin` shape and root -> `src/platforms/**` location are superseded by the lazy platform-module registry and private -> `@agent-device/platform-*` packages. Execution cuts over command-atomically; shared physical -> mechanics move only through ADR 0019's legal injected transition or after their last legacy user. +Completed process history is in git history. + +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** The Apple family and explicit `AppleOS` leaf axis remain accepted. +> The lazy platform-module registry and private `@agent-device/platform-*` packages supersede the shallow `PlatformPlugin` shape and root `src/platforms/**` location. +> Each command cuts over to the new execution path in one atomic step. +> Shared physical mechanics move only through ADR 0019's legal injected substrate transition or after their last legacy user. + +## Rules at a glance + +- Use one `apple` Platform with an `AppleOS` discriminant, not six Platform literals. +- Keep the OS-agnostic Apple engine in `packages/platform-apple/src/core/` and genuinely per-OS code in `packages/platform-apple/src/os//` leaves. +- Express per-OS capability differences as data keyed by `AppleOS`. +- Keep macOS as a distinct AppKit leaf. +- Do not flatten the tvOS focus-only interaction contract across OSes. +- Keep watchOS as an unsupported sentinel. Discovery never produces it. +- Internal family ownership must not leak into the public `ios`/`macos` leaf projection. +- As of 2026-08, no public-command coverage is claimed for visionOS or watchOS. Net-new visionOS spatial-input QA is deferred. ## Context -Apple support is modeled asymmetrically and is physically smeared across an `ios` directory that is really -the shared Apple engine. `Platform` carries `ios` and `macos` as separate literals, but tvOS is not a -platform at all — it is `platform: 'ios' + target: 'tv'`, with the OS name reconstructed late and lossily by -`resolveApplePlatformName` (`src/kernel/device.ts`). Meanwhile ~697 LOC of macOS code lives inside -`src/platforms/ios/`, `src/platforms/macos/devices.ts` is a 19-line stub that the iOS discovery imports (the -dependency arrow points backwards), and the Apple interactor in `src/core/interactors/apple.ts` reaches into -`platforms/ios`. A four-investigator survey found that ~85% of `platforms/ios` (the runner stack, -tool-provider, discovery, snapshot, screenshot, perf, debug-symbols) is already OS-agnostic, and the XCTest -runner already builds `ios | macos | tvos` from one Xcode project. Distinguishing or adding an Apple OS today -is therefore costly out of proportion to the actual work, and iPadOS/visionOS/watchOS are unmodeled. +Apple support uses an asymmetric model. +Physically, Apple support is spread across an `ios` directory, and that directory is actually the shared Apple engine. +`Platform` has separate `ios` and `macos` literals. +However, tvOS is not a platform: it is `platform: 'ios' + target: 'tv'`. +`resolveApplePlatformName` in `src/kernel/device.ts` reconstructs the OS name late and loses information. +Approximately 697 LOC of macOS code resides in `src/platforms/ios/`. +`src/platforms/macos/devices.ts` is a 19-line stub imported by iOS discovery, which reverses the intended dependency direction. +The Apple interactor in `src/core/interactors/apple.ts` also imports from `platforms/ios`. + +A survey by four investigators found approximately 85% of `platforms/ios` already OS-agnostic: + +- Runner stack. +- Tool-provider. +- Discovery. +- Snapshot. +- Screenshot. +- Perf. +- Debug-symbols. + +The XCTest runner already builds `ios | macos | tvos` from one Xcode project. +Thus, distinguishing or adding an Apple OS costs more than its actual implementation requires. +iPadOS, visionOS, and watchOS have no model. ## Decision -Model Apple OSes with an **`AppleOS` discriminant** (`ios | ipados | tvos | watchos | visionos | macos`) -under a single `apple` Platform — **not** six `Platform` literals. The OS-agnostic Apple engine consolidates -under `packages/platform-apple/src/core/`, with genuinely per-OS code in -`packages/platform-apple/src/os//` leaves; -the Apple plugin is the first instance of the platform-plugin registry (the platform axis of the -completed perfect-shape refactor; the registry and enforcement paths below are authoritative). -Per-OS capability differences become data keyed by `AppleOS`. The additive, -non-breaking `appleOs` discriminant — the groundwork for this — shipped in #896. +Model Apple OSes with an **`AppleOS` discriminant** (`ios | ipados | tvos | watchos | visionos | macos`) under one `apple` Platform. +Do not use six `Platform` literals. +Consolidate the OS-agnostic Apple engine under `packages/platform-apple/src/core/`. +Keep genuinely per-OS code in `packages/platform-apple/src/os//` leaves. +The Apple plugin is the first instance of the platform-plugin registry. +The registry and enforcement paths named below are authoritative. +Per-OS capability differences become data keyed by `AppleOS`. ## Alternatives Considered -- Promote each Apple OS to its own `Platform` literal: rejected. `DeviceTarget` (`mobile | tv | desktop`) is - already cross-platform (Android TV uses `target: 'tv'`), so a `tvos` literal collides with the form-factor - axis; it would also force the ~15 `isApplePlatform` and ~52 `macos` branch sites to enumerate six literals - and break the single-bucket `apple` capability/selector model that already works. -- Keep the status quo: rejected — the tvOS/macOS asymmetry, the mislabeled macOS code, and the lossy - target→OS inference persist, with no path to iPadOS/visionOS. -- Exclude macOS from the consolidation: rejected. macOS is already entangled — it builds via the same XCUITest - project and ~697 LOC of its code already lives inside `platforms/ios`. Excluding it would leave the - mislabel in place; including it as a distinct AppKit leaf normalizes it without homogenizing it. +- **Give each Apple OS its own `Platform` literal.** Rejected. + `DeviceTarget` (`mobile | tv | desktop`) already spans platforms. + Android TV uses `target: 'tv'`, so a `tvos` literal would conflict with the form-factor axis. + Approximately 15 `isApplePlatform` sites and approximately 52 `macos` branch sites would need to enumerate six literals. + This would also break the working single-bucket `apple` capability/selector model. +- **Keep the existing structure.** Rejected. + It retains tvOS/macOS asymmetry, mislabeled macOS code, and lossy target→OS inference without a path to iPadOS/visionOS. +- **Exclude macOS from consolidation.** Rejected. + macOS is already entangled. + It builds through the same XCUITest project, and approximately 697 LOC of its code resides in `platforms/ios`. + Exclusion would retain the incorrect label. + A distinct AppKit leaf normalizes macOS without making it the same as the other OSes. ## Consequences -Adding a first-class Apple OS becomes cheap: a leaf module plus a runner-profile row. iOS/iPadOS/tvOS/macOS -are mostly relocate-and-rename (the engine never needed to know which Apple OS it drives); visionOS is scoped -net-new work (XCUITest supports it — a profile row, a build case, `#if os(visionOS)`, a widened discovery -filter, plus real spatial-input QA); watchOS is an explicit **unsupported sentinel** because XCUITest cannot -drive watchOS UI. macOS stays a distinct AppKit leaf (its helper binary and menubar/desktop surface model are -preserved). The tvOS focus-only interaction contract (no coordinate `tap`) must not be flattened across OSes, -and snapshot fidelity is uneven (the deep-RN AX-server fallback is iOS-simulator-only). The internal -`Platform` collapse of `ios`+`macos` into `apple` was the last, highest-diff step; public leaf output -remains a separate compatibility projection. +Adding a first-class Apple OS becomes cheap: it needs a leaf module and a runner-profile row. +iOS/iPadOS/tvOS/macOS mostly require moves and renames because the engine never needed to know which Apple OS it drives. +visionOS is scoped net-new work. +XCUITest supports visionOS. +The work includes: + +- A profile row. +- A build case. +- `#if os(visionOS)`. +- A wider discovery filter. +- Real spatial-input QA. + +watchOS is an explicit **unsupported sentinel** because XCUITest cannot drive watchOS UI. +macOS remains a distinct AppKit leaf, and its helper binary and menubar/desktop surface model are preserved. +The tvOS focus-only interaction contract, with no coordinate `tap`, must not be flattened across OSes. +Snapshot fidelity is uneven. +The deep-RN AX-server fallback is iOS-simulator-only. +Public leaf output remains a separate compatibility projection. -This composes with ADR 0008 (the descriptor's capability facet) and ADR 0003. +This design composes with ADR 0008's descriptor capability facet and ADR 0003. Implementation status as of 2026-08: -- Shipped: the internal `Platform` collapse to `apple`; additive `appleOs` groundwork; the shared - Apple engine under `packages/platform-apple/src/core`; macOS leaf files under - `packages/platform-apple/src/os/macos`; a dedicated tvOS leaf under - `packages/platform-apple/src/os/tvos`; - direct internal imports to the Apple modules; the per-`AppleOS` runtime facts (including - `packages/platform-apple/src/gesture-facts.ts`; the former capability projection was retired after - its predicates moved into request-bound facts); the watchOS **unsupported sentinel** (reserved in the `AppleOS` type and interactor-rejected — - XCUITest cannot drive watchOS UI — `isSupportedAppleDeploymentLeaf`, the Apple interactor, and - gesture admission reject it — never produced by discovery); and visionOS profile/build/discovery - plus simulator-deployment evidence. -- Decision-only support boundary: visionOS discovery and simulator deployment are supported and - unit-tested (`packages/platform-apple/src/inventory-classification.ts` and - `packages/platform-apple/src/deployment/runtime.test.ts`); app deployment is also admitted for - CoreDevice-backed physical devices, while XCTest-backed physical deployment is unsupported and - push remains simulator-only. No public-command coverage is claimed for visionOS or watchOS. This - boundary is recorded here without creating a command-coverage manifest for either leaf. -- Retained compatibility: the public wire still emits `ios`/`macos` leaves through - `PUBLIC_PLATFORMS`; internal family ownership must not leak into that projection. +- Shipped: + - The internal `Platform` collapse to `apple`. + - Additive `appleOs` groundwork. + - The shared Apple engine under `packages/platform-apple/src/core`. + - macOS leaf files under `packages/platform-apple/src/os/macos`. + - A dedicated tvOS leaf under `packages/platform-apple/src/os/tvos`. + - Direct internal imports to the Apple modules. + - The per-`AppleOS` runtime facts, including `packages/platform-apple/src/gesture-facts.ts`. + The former capability projection was retired after its predicates moved into request-bound facts. + - The watchOS **unsupported sentinel**. + The `AppleOS` type reserves watchOS, and the interactor rejects it. + XCUITest cannot drive watchOS UI. + `isSupportedAppleDeploymentLeaf`, the Apple interactor, and gesture admission reject it. + Discovery never produces it. + - visionOS profile/build/discovery, plus simulator-deployment evidence. +- Decision-only support boundary: + - visionOS discovery and simulator deployment are supported and unit-tested in `packages/platform-apple/src/inventory-classification.ts` and `packages/platform-apple/src/deployment/runtime.test.ts`. + - App deployment is also admitted for CoreDevice-backed physical devices. + - XCTest-backed physical deployment is unsupported. + - Push remains simulator-only. + - No public-command coverage is claimed for visionOS or watchOS. + - This ADR records this boundary without a command-coverage manifest for either leaf. +- Retained compatibility: the public wire still emits `ios`/`macos` leaves through `PUBLIC_PLATFORMS`. + Internal family ownership must not leak into that projection. - Deferred: net-new visionOS spatial-input QA. -This ADR owns the Apple family/leaf decision. The Phase 3 platform-plugin umbrella (#972) is closed; -ADR 0019 now owns the staged package/runtime seam. Net-new visionOS spatial-input QA remains a -separate follow-up outside the consolidation. +This ADR owns the Apple family/leaf decision. +ADR 0019 now owns the staged package/runtime seam. +Net-new visionOS spatial-input QA remains a separate follow-up outside the consolidation. diff --git a/docs/adr/0010-error-system.md b/docs/adr/0010-error-system.md index eb24ec378d..04fe858d7d 100644 --- a/docs/adr/0010-error-system.md +++ b/docs/adr/0010-error-system.md @@ -3,62 +3,97 @@ - Status: accepted - Date: 2026-07-03 -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Existing normalization and -> cause-preservation rules remain accepted. When an operation and binding/resource cleanup both -> fail, the operation remains primary and cleanup is structured secondary diagnostic evidence; -> cleanup-only failure surfaces normally. +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Existing normalization and cause-preservation rules remain accepted. +> If an operation and binding/resource cleanup both fail, the operation remains primary. +> In that case, the cleanup failure is structured secondary diagnostic evidence. +> If only cleanup fails, the cleanup failure surfaces normally. + +## Rules at a glance + +- Every thrown error that a user or agent can reach is an `AppError`. Reserve bare `Error` for provably unreachable invariants. +- For input validation, throw `AppError('INVALID_ARGS', ...)`. To coerce an unknown caught error, use `asAppError(err, fallbackCode)`. +- Use the most specific `KnownAppErrorCode`. Put machine-dispatchable sub-classification in `details.reason`. +- If the per-code default hint would mislead, supply a hint. If the default suffices, omit the hint. +- When you wrap an error, preserve an existing hint and the cause chain. +- Every consumer surface must carry at least code, message, and hint. Rehydration helpers copy all fields. +- Failed requests always flush diagnostics. Messages must not embed raw stderr dumps or secrets. ## Context -The error system is centralized in `src/kernel/errors.ts` (`AppError`, `normalizeError`, -`defaultHintForCode`, `retriableForErrorCode`) but has ~1,200 construction sites across the CLI, -daemon, and platform layers. A July 2026 audit (code sweep + iOS/Android dogfooding) found the -kernel sound while call-site quality drifted: internal validation thrown as bare `Error` -(surfacing as `UNKNOWN` with a misleading hint), semantically wrong codes (`COMMAND_FAILED` for -busy devices, `INVALID_ARGS` for expired server-side resources), missing hints on the most common -agent failures (selector/ref misses), and consumers that drop fields (MCP tool errors carried only -`message`). Nothing documented the conventions, so each new site re-decided them. +The error system is centralized in `packages/kernel/src/errors.ts` (`AppError`, `normalizeError`, `defaultHintForCode`, `retriableForErrorCode`). +However, approximately 1,200 construction sites exist across the CLI, daemon, and platform layers. +A July 2026 audit combined a code sweep with iOS/Android dogfooding. +It found the kernel sound, but call-site quality had drifted: + +- Internal validation threw bare `Error`, which surfaced as `UNKNOWN` with a misleading hint. +- Some codes were semantically wrong, such as `COMMAND_FAILED` for busy devices and `INVALID_ARGS` for expired server-side resources. +- The most common agent failures, selector/ref misses, had no hints. +- Some consumers dropped fields; for example, MCP tool errors carried only `message`. + +Nothing documented the conventions, so each new site decided them again. ## Decision -1. **Every thrown error a user or agent can reach is an `AppError`.** Bare `throw new Error(...)` - is reserved for provably unreachable invariants. Input validation — including daemon-side - decoding of command input — throws `AppError('INVALID_ARGS', ...)`. Coercion of unknown caught - errors uses `asAppError(err, fallbackCode)`, which preserves the cause chain; do not hand-roll - `err instanceof AppError ? err : new AppError(...)`. -2. **Code selection.** Use the most specific `KnownAppErrorCode`; `COMMAND_FAILED` is for genuine - runtime failures of a well-formed request, never a catch-all for capability gaps - (`UNSUPPORTED_OPERATION`), contention (`DEVICE_IN_USE`, the only code retriable by default; - the cross-worktree device-claim path overrides it to `retriable: false`), or ambiguity - (`AMBIGUOUS_MATCH`). New codes are added to the union deliberately; machine-dispatchable - sub-classification rides in `details.reason` (the lease registry is the model). -3. **Hints answer "what should the agent run next".** A hint is required wherever the per-code - default from `defaultHintForCode` would mislead; it is omitted where the default suffices — - mass-adding boilerplate hints is worse than the default. Shared failure modes get shared hint - constants next to the code that detects them (`selectorFailureHint`, `STALE_REF_HINT` in - `packages/selectors/src/internal/resolve.ts`; `resolveIosDevicectlHint`; `bootFailureHint`), not copy-pasted - strings. Re-wraps preserve an existing hint rather than clobbering it. -4. **Wrapping external tool failures.** Prefer `exec.ts` errors as-is. A hand-rolled wrap of an - `allowFailure` result must carry `{ stdout, stderr, exitCode, processExitError: true }` so - `normalizeError` can surface the first meaningful stderr line instead of "tool exited with - code N". -5. **The wire error contract** is `code`, `message`, `hint`, `details` (redacted), `diagnosticId`, - `logPath`, plus optional typed signals `retriable` / `supportedOn` — the fields are absent - unless confidently known. Every consumer surface (CLI human, CLI `--json`, SDK, MCP, events - timeline) must carry code + message + hint at minimum; rehydration helpers - (`throwDaemonError`, `toDaemonHttpRpcError`) copy all fields, and `NormalizedError` hoists the - typed signals so `--json` consumers see them. -6. **Observability.** Failed requests always flush diagnostics: `diagnosticId` + ndjson `logPath` - accompany every error (daemon-side under the session state dir, client-side under - `~/.agent-device/logs`). Messages must not embed raw stderr dumps or secrets — structured - context belongs in `details`, which is redacted at normalize/write time. +1. **Every thrown error a user or agent can reach is an `AppError`.** + Bare `throw new Error(...)` is reserved for provably unreachable invariants. + Input validation, including daemon-side decoding of command input, throws `AppError('INVALID_ARGS', ...)`. + To coerce an unknown caught error, use `asAppError(err, fallbackCode)`, which preserves the cause chain. + Do not hand-roll `err instanceof AppError ? err : new AppError(...)`. +2. **Code selection.** Use the most specific `KnownAppErrorCode`. + `COMMAND_FAILED` is for genuine runtime failures of a well-formed request. + Never use `COMMAND_FAILED` as a catch-all for these cases: + - Capability gaps. Use `UNSUPPORTED_OPERATION`. + - Contention. Use `DEVICE_IN_USE`. + - Ambiguity. Use `AMBIGUOUS_MATCH`. + `DEVICE_IN_USE` is the only code that is retriable by default. + The cross-worktree device-claim path overrides that default to `retriable: false`. + Add new codes to the union deliberately. + Machine-dispatchable sub-classification goes in `details.reason`; the lease registry is the model. +3. **Hints answer "what should the agent run next".** + If the per-code default from `defaultHintForCode` would mislead, a hint is required. + If the default suffices, omit the hint. + Boilerplate hints added in bulk are worse than the default. + Shared failure modes get shared hint constants next to the code that detects them, not copy-pasted strings: + - `selectorFailureHint` and `STALE_REF_HINT` in `packages/selectors/src/internal/resolve.ts`. + - `resolveIosDevicectlHint`. + - `bootFailureHint`. + A re-wrap preserves an existing hint instead of overwriting it. +4. **Wrapping external tool failures.** Prefer `exec.ts` errors as they are. + A hand-rolled wrap of an `allowFailure` result must carry `{ stdout, stderr, exitCode, processExitError: true }`. + Thus, `normalizeError` can surface the first meaningful stderr line instead of "tool exited with code N". +5. **The wire error contract** contains: + - `code`. + - `message`. + - `hint`. + - `details` (redacted). + - `diagnosticId`. + - `logPath`. + - Optional typed signals `retriable` / `supportedOn`. + The typed signals are absent unless they are confidently known. + Every consumer surface must carry at least code, message, and hint: + - CLI human output. + - CLI `--json`. + - SDK. + - MCP. + - Events timeline. + Rehydration helpers (`throwDaemonError`, `toDaemonHttpRpcError`) copy all fields. + `NormalizedError` hoists the typed signals, so `--json` consumers see them. +6. **Observability.** Failed requests always flush diagnostics. + A `diagnosticId` and an ndjson `logPath` accompany every error. + Daemon-side diagnostics are under the session state dir. + Client-side diagnostics are under `~/.agent-device/logs`. + Messages must not embed raw stderr dumps or secrets. + Structured context belongs in `details`, which is redacted at normalize/write time. ## Consequences -- Agents can branch on `code`, retry on `retriable`, and follow `hint` without parsing prose; - wording can improve without breaking consumers (except strings pinned by help-text guarantees — - update those tests in lockstep). -- The known-code union stays small and meaningful; the widened `(string & {})` type still lets - runner/daemon codes pass through, so SDK consumers keep a `default` branch. -- New call sites have a documented bar: right code, contextual message naming the target, hint - only when the default misleads, cause preserved. +- Agents can branch on `code`, retry on `retriable`, and follow `hint` without prose parsing. + Wording can improve without breaking consumers, except for strings that help-text guarantees pin. + If you change one of those strings, update its tests in the same change. +- The known-code union stays small and meaningful. + The widened `(string & {})` type still lets runner/daemon codes pass through, so SDK consumers keep a `default` branch. +- New call sites have a documented bar: + - The correct code. + - A contextual message that names the target. + - A hint only when the default misleads. + - A preserved cause. diff --git a/docs/adr/0011-interaction-guarantee-contract.md b/docs/adr/0011-interaction-guarantee-contract.md index 3b0a904140..810dd0ad0e 100644 --- a/docs/adr/0011-interaction-guarantee-contract.md +++ b/docs/adr/0011-interaction-guarantee-contract.md @@ -4,63 +4,70 @@ Accepted (implemented through Layer 3, 2026-07-04: #1080, #1082–#1086, #1091, #1092; ambiguity contract amended 2026-08-07) +Completed process history is in git history. + +## Rules at a glance + +- Classify every dispatch-path × guarantee cell in the typed registry. +- Link each `gap:` waiver to a tracking issue. Keep command-scoped claims within the commands that the path dispatches. +- Give each guarantee exactly one TypeScript home that all TypeScript paths import. Prove the parity of runner-side Swift twins through golden fixture tables. +- Prove every non-waived cell through a tagged contract scenario. +- Delegation-on-error does not prove success-path parity. +- On the native-ref path, before `tapTarget`/`fillTarget`, check offscreen, occlusion, and non-hittable against the node from the daemon snapshot. +- The Apple runner does not resend a mutating command whose first send may have run. +- Multiple mutating selector matches collapse only if they form one ancestor–descendant chain and every member resolves to the same actionable node. Otherwise, the mutation fails with `AMBIGUOUS_MATCH`. Maestro's explicit expected-point/non-hittable compatibility path stays separate. +- Keep synthesized iOS gesture policy outside the element dispatch-path matrix. + ## Context -Interaction commands (`press`/`click`/`fill`/`longpress`, and by extension the -read/wait surfaces built on the same resolution machinery) reach the device -through several dispatch paths, each of which chose its latency-vs-semantics -trade-offs independently: +Interaction commands reach the device through several dispatch paths. +These commands are `press`/`click`/`fill`/`longpress` and, by extension, the read/wait surfaces built on the same resolution machinery. +Each path independently selected its latency-vs-semantics trade-offs: | Path | What it is | Why it exists | | --- | --- | --- | | `runtime-selector` | daemon tree capture → `resolveSelectorChain` → guards → coordinate tap | full semantics | | `runtime-ref` | session snapshot → ref lookup → guards → coordinate tap | full semantics | | `target-drag` | independently resolve source + destination → guards → one pointer plan | dual-endpoint drag semantics | -| `direct-ios-selector` | selector sent to the XCTest runner, which queries and taps natively | saves a full snapshot round trip | -| `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | saves resolution round trips | +| `direct-ios-selector` | selector sent to the XCTest runner, which queries and taps natively | avoids a full snapshot round trip | +| `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | avoids resolution round trips | | `coordinate` | raw x/y tap | escape hatch; semantics intentionally minimal | -| replay/maestro variants | `allowNonHittableCoordinateFallback`, replay-heal | third-party compat | - -Over one week of fixes, every interaction bug we shipped or caught was the same -shape — a guarantee enforced on one path and silently absent on a sibling: - -- off-screen refusal existed for `@ref` targets but not selector targets, and - not in the runner's native tap (Bluesky closed drawer: `Tapped (-161, 265)` - reported as success; fixed in #1075); -- `--verify` evidence attached on `press @ref` but was dropped by - `fill @ref`'s hand-built response branch (#1064 review); -- adb failure classification enriched thrown errors but not - `allowFailure`-result errors, and wrapped `exec` but not semantic provider - methods (#1067 review); -- the user's wait budget bounded prepare/replay/snapshot request envelopes but - not `wait` itself, and `wait` sat on the daemon-reset timeout path while - `snapshot` — same failure mode — preserved the daemon (#1075). - -The guarantees themselves are small and well-implemented. What is missing is -any structure that *knows the full set of paths and the full set of -guarantees* and can therefore notice an unfilled cell. Reviews catch cells -only when a reviewer happens to hold both axes in their head; dogfooding -catches them one production incident at a time. +| replay/maestro variants | `allowNonHittableCoordinateFallback`, replay-heal | third-party compatibility | + +Over one week of fixes, every interaction bug that we shipped or caught had the same pattern. +One path enforced a guarantee, and a sibling path silently did not have it: + +- Off-screen refusal existed for `@ref` targets, but not for selector targets and not in the runner's native tap. + A closed Bluesky drawer reported `Tapped (-161, 265)` as success. #1075 fixed this. +- `press @ref` attached `--verify` evidence. + The hand-built response branch of `fill @ref` dropped it (#1064 review). +- adb failure classification enriched thrown errors, but not errors from `allowFailure` results. + The classification wrapped `exec`, but not semantic provider methods (#1067 review). +- The user's wait budget bounded prepare/replay/snapshot request envelopes, but not `wait` itself. + Also, `wait` was on the daemon-reset timeout path, while `snapshot`, with the same failure mode, preserved the daemon (#1075). + +The guarantees themselves are small and well implemented. +The missing part is a structure that knows the full set of paths and the full set of guarantees, and can therefore detect an unfilled cell. +Reviews find such a cell only if a reviewer happens to know both axes at the same time. +Dogfooding finds such cells one production incident at a time. ## Decision -Make the path × guarantee matrix a first-class, machine-checked artifact, with -three enforcement layers. Types enforce **completeness** of declarations, -shared implementations prevent **drift**, and generated test coverage enforces -**truth**. +Make the path × guarantee matrix a first-class, machine-checked artifact with three enforcement layers. +Types enforce **completeness** of declarations. +Shared implementations prevent **drift**. +Generated test coverage enforces **truth**. ### Layer 1 — Declare: the matrix as a typed registry with a gate -Layer 1 is an **honesty/completeness gate, not a truth gate**: it proves that -every path has declared a stance on every guarantee and that the referenced -implementations exist. It does not prove the declared behavior — behavioral -parity only starts once the fixture tables (Layer 2) and contract scenarios -(Layer 3) land. Landing Layer 1 alone still changes the failure mode: an -unwatched cell becomes impossible; an unproven cell is at least a visible, -owned claim. +Layer 1 is an **honesty/completeness gate, not a truth gate**. +It proves that every path declares a stance on every guarantee and that the referenced implementations exist. +It does not prove the declared behavior. +Behavioral parity starts only after the fixture tables (Layer 2) and the contract scenarios (Layer 3) are in place. +Layer 1 alone still changes the failure mode: an unwatched cell becomes impossible. +An unproven cell is at least a visible claim that has an owner. -`packages/contracts/src/interaction-guarantees.ts` declares both axes and requires -every cell to be classified: +`packages/contracts/src/interaction-guarantees.ts` declares both axes and requires classification of every cell: ```ts export const INTERACTION_GUARANTEES = [ @@ -75,19 +82,22 @@ export const INTERACTION_GUARANTEES = [ ] as const; ``` -Cells may be **command-scoped** via `appliesTo` when a guarantee only exists -for a subset of the path's commands — e.g. `verifyEvidence` applies to -`press`/`click`/`fill` but not `longpress`, and claiming it path-wide would -overstate coverage. The gate rejects `appliesTo` entries naming commands the -path does not dispatch, and rejects redundant full-coverage lists. +If a guarantee exists only for a subset of the path's commands, cells may be **command-scoped** through `appliesTo`. +For example, `verifyEvidence` applies to `press`/`click`/`fill`, but not to `longpress`. +A path-wide claim would overstate coverage. +The gate rejects an `appliesTo` entry that names a command that the path does not dispatch. +The gate also rejects a redundant `appliesTo` list that covers all commands of the path. + +`responseConstruction` and `responseIdentity` are deliberately separate. +"Use one shared construction site" and "which identity fields this path can provide" have different closure strategies. +The former is a single Layer-2 refactor. +The latter is per-path capability work. +`errorTaxonomy` is expected to split the same way later: -`responseConstruction` and `responseIdentity` are deliberately separate: "use -one shared construction site" and "which identity fields this path can -provide" have different closure strategies (the former is a single Layer-2 -refactor; the latter is per-path capability work). `errorTaxonomy` is expected -to split the same way later — stable codes/fallback classification vs rich -selector diagnostics and hints — because direct runner paths can close codes -long before full diagnostics. +- Stable codes and fallback classification. +- Rich selector diagnostics and hints. + +The reason is that direct runner paths can close codes long before full diagnostics. ```ts @@ -107,171 +117,168 @@ export const INTERACTION_DISPATCH_PATHS: Record< > = { /* every path, every cell */ }; ``` -Because `guarantees` is a `Record` over the guarantee union, **tsc fails** the -moment someone adds a guarantee without classifying it for every path, or adds -a path without classifying every guarantee. A unit gate -(`interaction-guarantees.test.ts`) additionally checks that: +`guarantees` is a `Record` over the guarantee union. +Thus, **tsc fails** if someone adds a guarantee without a classification for every path. +tsc also fails if someone adds a path without a classification of every guarantee. +The unit gate `interaction-guarantees.test.ts` also checks that: + +- Every `via` string resolves to a real exported symbol, so declarations cannot become fiction. +- Every `parityTable` names an existing fixture file. +- Every `waived` reason is non-empty. +- Every `gap:` waiver carries a `trackingIssue`. -- every `via` string resolves to a real exported symbol (declarations cannot - rot into fiction); -- every `parityTable` names an existing fixture file; -- every `waived` reason is non-empty, and every `gap:` waiver carries a - `trackingIssue` — waivers must be owned, not merely visible. One umbrella - tracking issue with sub-issues split off as work is scheduled is - sufficient; the gate enforces the link, not the granularity. +Waivers must have an owner, not only visibility. +One umbrella tracking issue is sufficient, with sub-issues split off when work is scheduled. +The gate enforces the link, not the granularity. -This is the same "make the gap declare itself" pattern already proven in this -repo by `scripts/integration-progress-model.ts` (which caught the unclassified -`--verify` flag on #1064) and the cross-command apple-leak guard from the -platform consolidation (ADR 0009). +This repository already proved the same "make the gap declare itself" pattern in two places: + +- `scripts/integration-progress-model.ts`, which caught the unclassified `--verify` flag on #1064. +- The cross-command apple-leak guard from the platform consolidation (ADR 0009). ### Layer 2 — Share: one implementation per rule, on both sides of the wire -Each guarantee has exactly one home that all TS paths import (most already -exist after #1075: `isNodeVisibleOnScreen`, `accumulateDisambiguationCandidate`, -`isSnapshotNodeInteractionBlocked`, `describeResolvedNode`, -`interactionResultExtra`, `reconcileNonHittableHintWithEvidence`). The -registry's `via` fields point at them; fallow's duplication gate keeps -re-implementations from creeping back. - -Rules that must run **runner-side** (the direct iOS path cannot see the daemon -tree) get pure-function Swift twins operating on plain geometry — no -`XCUIElement` — e.g. `TapPointPolicy.isAllowed(elementFrame:windowFrame:)`. -Parity is enforced by **golden fixture tables**: JSON files under -`contracts/fixtures/` consumed by three test suites — - -1. vitest asserts the TS rule over the table; -2. the runner's Swift unit tests (already compiled in CI by "Swift Runner Unit - Compile") assert the Swift twin over the same table; -3. the provider harness's fake runner derives its behavior from the same - table, so integration scenarios exercise the real contract rather than a - hand-written approximation. - -Drift between TS and Swift then turns CI red on whichever side changed, -without needing a simulator. - -`contracts/fixtures/dispatch-disclosure.json` is the table for -`AppErrorDetails.dispatched` on interaction failures: one row per producer -event, each with the value it must leave. The field has two values: `no` (the -operation never reached the device, or the command is a read, so a resend is -safe) and `unknown` (it may have landed, so observe before resending). Only a -producer that refuses before dispatch (target resolution, admission, a runner -pre-send refusal) may say `no`; no producer can prove execution on its failure -path, so there is no third value. - -Each request owns one dispatch ledger. Every bound runtime operation declares -its effect once, in `RUNTIME_OPERATION_EFFECTS`, a record over the runtime -operation keys, so an operation cannot bind without one. The request binding -records each `mutates` operation in the request's ledger when its send -returns, so no bound runtime operation sends a mutation outside the ledger. -Android ANR and blocking-dialog recovery is outside the ledger: its -`AndroidObservationAdapter.tap` and `openApp` calls reach the device without a -record. A nested request (a -batch step, a replay action, a delegated `find` click or fill) records into a -ledger of its own and moves its sends into its parent's. - -The request router discloses around every routed command. Once the ledger -holds a sent mutation, a failure is `unknown` with the sent count in -`details.dispatchedSteps`, whatever produced it and whatever the command -declares: a wrong `no` makes a consumer resend an action that already ran, -while a wrong `unknown` only costs an observation. Before that, the registry's -`recordingEffect` decides. For `'mutates-app'` the router keeps a producer's -value and fills `unknown` for a failure no producer classified. For -`'observes-app'` it sets `no` over any producer value. A command with no -declared effect passes through. - -`no` for a read means that a resend repeats no app-visible action. The -registry declares `record`, `trace`, and `perf` as `'observes-app'`, although -their recorder and profiler controls reach the device: no registry trait -separates a pure read from a device control, and those controls are declared -`repeatable` at the operation level, because the device refuses or ignores a -repeat. A producer that runs several device inputs inside one bound operation -(an Android double tap, an iOS press series) counts them in -`details.dispatchedSteps` itself, because the ledger sees one operation. - -Each row names its driver file by id prefix, and that file drives the real -producer. - -The Apple runner does not resend a mutating command whose first send may have -run: a restart resends only a command the first attempt provably did not write, -or a read-only one. A mutation whose reply stays lost (the runner dies -mid-command, or status recovery finds neither a retained result nor a runner -answer) fails with `reason: runner_reply_lost` and `dispatched: unknown` (rows -`ios-runner.transport.written-then-lost`, -`ios-runner.status.completed-without-retained-reply`, -`ios-runner.status.accepted`, `ios-runner.status.started`, -`ios-runner.status.notAccepted`, `ios-runner.status.probe-failed`, and -`ios-runner.status.unavailable`). The `ios-runner.status.failed*` rows carry the -runner's own answer instead. A read keeps its transport error and is resent. - -Remaining gaps: a failure before the router's locked scope (session -resolution, lock acquisition, lease and daemon-policy admission) never reaches -the disclosure and carries no `dispatched`. It sends nothing, but a consumer -must read the absent field as `unknown`. - -`runBatch` reports `no` only when every executed step is a command declared -`observes-app` (a read); any other executed step, including one with no -declared `recordingEffect`, makes the batch failure `unknown` for every caller, -including one that supplies its own `invoke`. - -For `responseFields`, one `buildInteractionResponseData(...)` becomes the only -construction site for interaction response payloads (this deletes the class of -bug where `fill @ref` rebuilt its response by hand and dropped `evidence`). A -small guard test — repo-precedented — fails if an interaction handler contains -a hand-rolled `responseData = {` literal. +Each guarantee has exactly one home that all TypeScript paths import. +Most of these homes already exist after #1075: + +- `isNodeVisibleOnScreen`. +- `accumulateDisambiguationCandidate`. +- `isSnapshotNodeInteractionBlocked`. +- `describeResolvedNode`. +- `interactionResultExtra`. +- `reconcileNonHittableHintWithEvidence`. + +The registry's `via` fields point at them. +Fallow's duplication gate prevents re-implementations from coming back. + +Some rules must run **runner-side**, because the direct iOS path cannot see the daemon tree. +These rules get pure-function Swift twins that operate on plain geometry, without `XCUIElement`. +`TapPointPolicy.isAllowed(elementFrame:windowFrame:)` is one example. +**Golden fixture tables** enforce parity. +These tables are JSON files under `contracts/fixtures/`, and three test suites consume them: + +1. Vitest asserts the TypeScript rule against the table. +2. Runner Swift unit tests assert the Swift twin against the same table. + CI already compiles them through "Swift Runner Unit Compile". +3. The provider harness's fake runner derives its behavior from the table. + Integration scenarios therefore exercise the real contract instead of a manual approximation. + +Then, a TypeScript or Swift drift turns CI red on the side that changed, without a simulator. + +`contracts/fixtures/dispatch-disclosure.json` is the table for `AppErrorDetails.dispatched` on interaction failures. +It has one row for each producer event, and each row has the value that the producer must leave. +The field has two values: + +- `no`: the operation never reached the device, or the command is a read, so a resend is safe. +- `unknown`: the operation may have reached the device, so observe before a resend. + +Only a producer that refuses before dispatch (target resolution, admission, a runner pre-send refusal) may report `no`. +No producer can prove execution on its failure path, so there is no third value. + +Each request owns one dispatch ledger. +Every bound runtime operation declares its effect once, in `RUNTIME_OPERATION_EFFECTS`, a record over the runtime operation keys. +Thus, an operation cannot bind without an effect. +When the send of a `mutates` operation returns, the request binding records the operation in the request's ledger. +Thus, no bound runtime operation sends a mutation outside the ledger. +Android ANR and blocking-dialog recovery is outside the ledger. +Its `AndroidObservationAdapter.tap` and `openApp` calls reach the device without a record. +A nested request records into a ledger of its own and moves its sends into its parent's ledger. +A batch step, a replay action, and a delegated `find` click or fill are nested requests. + +The request router discloses around every routed command. +After the ledger holds a sent mutation, a failure is `unknown`, with the sent count in `details.dispatchedSteps`. +This applies whatever produced the failure and whatever the command declares. +The reason: a wrong `no` makes a consumer resend an action that already ran, while a wrong `unknown` only costs an observation. +Before the ledger holds a sent mutation, the registry's `recordingEffect` decides: + +- For `'mutates-app'`, the router keeps a producer's value. + If no producer classified the failure, the router fills `unknown`. +- For `'observes-app'`, the router sets `no` over any producer value. +- A command with no declared effect passes through. + +For a read, `no` means that a resend repeats no app-visible action. +The registry declares `record`, `trace`, and `perf` as `'observes-app'`, although their recorder and profiler controls reach the device. +There are two reasons: + +- No registry trait separates a pure read from a device control. +- `RUNTIME_OPERATION_EFFECTS` declares those controls `repeatable` at the operation level, because the device refuses or ignores a repeat. + +A producer can run several device inputs inside one bound operation, such as an Android double tap or an iOS press series. +That producer counts the inputs in `details.dispatchedSteps` itself, because the ledger sees one operation. + +Each row names its driver file by id prefix, and that file drives the real producer. + +The Apple runner does not resend a mutating command whose first send may have run. +A restart resends only a command that the first attempt provably did not write, or a read-only command. +A mutation reply can stay lost: the runner dies mid-command, or status recovery finds neither a retained result nor a runner answer. +In that case, the mutation fails with `reason: runner_reply_lost` and `dispatched: unknown`. +The rows for this case are: + +- `ios-runner.transport.written-then-lost`. +- `ios-runner.status.completed-without-retained-reply`. +- `ios-runner.status.accepted`. +- `ios-runner.status.started`. +- `ios-runner.status.notAccepted`. +- `ios-runner.status.probe-failed`. +- `ios-runner.status.unavailable`. + +The `ios-runner.status.failed*` rows carry the runner's own answer instead. +A read keeps its transport error and is resent. + +Remaining gaps: a failure before the router's locked scope (session resolution, lock acquisition, lease and daemon-policy admission) never reaches the disclosure and carries no `dispatched`. +Such a failure sends nothing, but a consumer must read the absent field as `unknown`. + +`runBatch` reports `no` only if every executed step is a command declared `observes-app` (a read). +Any other executed step makes the batch failure `unknown`, including a step with no declared `recordingEffect`. +This applies to every caller, including a caller that supplies its own `invoke`. + +For `responseFields`, one `buildInteractionResponseData(...)` becomes the only construction site for interaction response payloads. +This deletes the class of bug where `fill @ref` rebuilt its response by hand and dropped `evidence`. +A small guard test, with precedent in this repository, fails if an interaction handler contains a hand-rolled `responseData = {` literal. ### Layer 3 — Prove: a contract suite generated from the registry -`test/integration/interaction-contract/` holds table-driven scenarios: fixture -tree × command × forced path. The fixture trees are the real shapes that found -this week's bugs, kept permanently: +`test/integration/interaction-contract/` holds table-driven scenarios: fixture tree × command × forced path. +The fixture trees are the real shapes that found this week's bugs, and the suite keeps them permanently: -- closed drawer (all candidates off-screen) → `offscreen_selector`/`offscreen_ref`; -- stale or unknown `@ref` (errorTaxonomy) → `ref_not_found` beside the stale-ref hint; a listed - `@ref` whose node has no usable centre (missing, non-finite, or negative bounds), or a selector - target without one → `target_bounds_invalid`; a listed `@ref` with no label to wait on or scope - by → `ref_unlabeled`; -- drawer item + visible twin (ambiguous on/off-screen) → visible candidate wins; -- edge-grazing container (0.07 px viewport overlap, center off-screen) → still refused; -- covered node → occlusion refusal; -- non-hittable target ± `--verify` → hint present / suppressed by evidence; -- stripped-root tree (no Application/Window) → safe-default visibility. +- Closed drawer (all candidates off-screen) → `offscreen_selector`/`offscreen_ref`. +- Stale or unknown `@ref` (errorTaxonomy) → `ref_not_found` beside the stale-ref hint. +- A listed `@ref` whose node has no usable centre (missing, non-finite, or negative bounds), or a selector target without one → `target_bounds_invalid`. +- A listed `@ref` with no label to wait on or to scope by → `ref_unlabeled`. +- Drawer item + visible twin (ambiguous on/off-screen) → the visible candidate wins. +- Edge-grazing container (0.07 px viewport overlap, center off-screen) → still refused. +- Covered node → occlusion refusal. +- Non-hittable target ± `--verify` → hint present / suppressed by evidence. +- Stripped-root tree (no Application/Window) → safe-default visibility. -Path forcing is explicit (`AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`, -test-only) so cases stay stable when path-selection heuristics change. +Path forcing is explicit and test-only: `AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`. +Thus, cases stay stable when path-selection heuristics change. -The gate closes the loop: it walks the Layer-1 registry and fails when any -non-waived cell has no contract case tagged for it. Coverage of the matrix is -therefore by construction, not by reviewer memory. +The gate closes the loop. +It walks the Layer-1 registry and fails if any non-waived cell has no contract case tagged for it. +Coverage of the matrix therefore comes from construction, not from reviewer memory. ### Closing the gaps: a hybrid strategy, not one answer -The acknowledged gaps close by different mechanisms depending on what the -guarantee needs: - -- **Runner-side parity for cheap geometry-local rules** (offscreen / - tappable-frame on the direct iOS path). These are pure frame math, provable - with golden tables, and keep the fast path fast. -- **Delegation-on-error for semantic and rich-runtime cases** - (`ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, non-hittable - refusal): the fast path fails cheaply, and the runtime path supplies - disambiguation and full diagnostics only when needed. **Delegation-on-error - is not success-path parity**: it cannot catch the case where XCTest finds - one hittable candidate that runtime rules would refuse or rank differently. - Those cells stay `gap:` waivers until parity tables or contract scenarios - prove the success path too. -- **A shared runtime preflight for the native-ref path**: the ref came from a - daemon snapshot, so the node is already in hand — check offscreen / - occlusion / non-hittable against it *before* calling - `tapTarget`/`fillTarget`. A backend fast path can silently "succeed", so - delegation-on-error would never trigger there. +The acknowledged gaps close by different mechanisms, which depend on what the guarantee needs: + +- **Runner-side parity for cheap geometry-local rules** (offscreen/tappable-frame on the direct iOS path). + These rules are pure frame math, golden tables can prove them, and they keep the fast path fast. +- **Delegation-on-error for semantic and rich-runtime cases** (`ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, non-hittable refusal). + The fast path fails cheaply. + The runtime path supplies disambiguation and full diagnostics only when they are necessary. + **Delegation-on-error is not success-path parity.** + It cannot catch the case where XCTest finds one hittable candidate that runtime rules would refuse or rank differently. + Those cells stay `gap:` waivers until parity tables or contract scenarios prove the success path too. +- **A shared runtime preflight for the native-ref path.** The ref came from a daemon snapshot, so the node is already available. + Check offscreen, occlusion, and non-hittable against that node *before* the call to `tapTarget`/`fillTarget`. + A backend fast path can silently "succeed", so delegation-on-error would never start there. ### Timeout policy joins the descriptor registry -The `wait` timeout bug existed because request-envelope budgets and -on-timeout daemon policy lived in two hand-maintained lists in the client -(`isExplicitTimeoutCommand`, `shouldResetDaemonAfterRequestTimeout`). Both are -replaced by declarations on the command descriptors (ADR 0008 registry): +The `wait` timeout bug existed because two hand-maintained lists in the client held request-envelope budgets and on-timeout daemon policy. +The two lists were `isExplicitTimeoutCommand` and `shouldResetDaemonAfterRequestTimeout`. +Declarations on the command descriptors (ADR 0008 registry) replace both lists: ```ts timeoutPolicy: { @@ -280,169 +287,178 @@ timeoutPolicy: { } ``` -with a completeness gate over all public commands. Read-only polling commands -declare `preserve-daemon`; the client derives the envelope from the declared -budget source instead of special-casing command names. Since #1105 the -capture-resolving interaction commands (`click`, `fill`, `longpress`, `press`, -`type`, `get`, `is`) also declare `preserve-daemon`: their dominant timeout -mode is the same blocked accessibility capture as `snapshot`, and resetting -the daemon there destroyed every healthy app session the daemon owned. +A completeness gate covers all public commands. +Read-only polling commands declare `preserve-daemon`. +The client derives the envelope from the declared budget source, instead of special cases for command names. +Since #1105, the capture-resolving interaction commands also declare `preserve-daemon`: + +- `click`. +- `fill`. +- `longpress`. +- `press`. +- `type`. +- `get`. +- `is`. + +Their dominant timeout mode is the same blocked accessibility capture as `snapshot`. +A daemon reset there destroyed every healthy app session that the daemon owned. ## Consequences -- Adding a dispatch path or a guarantee becomes a *forced* whole-matrix - decision: tsc will not compile an unclassified cell, and the gate will not - pass an untested one. Silent erosion is structurally impossible; explicit - waivers remain possible but visible and linkable to issues. -- Fast paths keep their latency wins. Their divergence is priced and - documented instead of discovered on-device. -- The Swift/TS split stops being a parity blind spot: golden tables are the - single source of truth for cross-language rules, and the fake runner stops - being a second, drifting implementation. -- Initial cost is mostly classification honesty: the first registry will - contain `waived('gap: ...')` cells (e.g. the maestro non-hittable fallback - intentionally waives `offscreen`), which is the point — the debt becomes a - diffable list instead of folklore. -- The registry doubles as documentation input for help/skill output ("what - press guarantees"), which matters for small-model agents that only read the - contract, never the code. +- A new dispatch path or a new guarantee forces a whole-matrix decision. + tsc does not compile an unclassified cell, and the gate does not pass an untested cell. + Silent erosion is structurally impossible. + Explicit waivers remain possible, but they are visible and can link to issues. +- Fast paths keep their latency wins. + Their divergence is priced and documented, instead of discovered on-device. +- The Swift/TypeScript split stops being a parity blind spot. + Golden tables are the single source of truth for cross-language rules. + The fake runner stops being a second implementation that drifts. +- The initial cost is mostly classification honesty. + The first registry contains `waived('gap: ...')` cells, and that is the purpose. + For example, the maestro non-hittable fallback intentionally waives `offscreen`. + The debt becomes a diffable list instead of folklore. +- The registry is also documentation input for help/skill output ("what press guarantees"). + This matters for small-model agents that only read the contract, never the code. ### 2026-08-07 amendment: mutating ambiguity fails fast -Element-14 realized the matrix's previously owned success-path gap: four exact -`Team Standup` matches in distinct accessibility subtrees reached ordinary -runtime selector resolution, whose visible/depth/area ranking silently chose a -different semantic target. Mutating selectors no longer use geometry to choose -among distinct subtrees. - -The replacement contract is structural. Multiple matches collapse only when -all matches form one ancestor–descendant chain and every member resolves to the -same actionable node. Otherwise the mutation fails with `AMBIGUOUS_MATCH`, a -bounded list of snapshot candidate lines, and a partial ref frame generation so -the caller can retry one listed candidate immediately. On the direct XCTest path -this applies to mutating dispatches only: they count all raw exact matches before -hittability can select a winner and delegate multiple matches to the runtime -classifier. Reads (`querySelector`, and so `get`/`is`/`wait`) keep the prior rule -— prefer the single hittable match, ambiguous only when hittable matches compete -— because a read has no side effect to guard and failing it closed would turn a -decorative duplicate into an error. Maestro's explicit expected-point / -non-hittable compatibility path remains intentionally separate. +Element-14 made the matrix's previously owned success-path gap real. +Four exact `Team Standup` matches in distinct accessibility subtrees reached ordinary runtime selector resolution. +Its visible/depth/area ranking silently chose a different semantic target. +Mutating selectors no longer use geometry to choose among distinct subtrees. + +The replacement contract is structural. +Multiple matches collapse only if all matches form one ancestor–descendant chain and every member resolves to the same actionable node. +Otherwise, the mutation fails with `AMBIGUOUS_MATCH`, a bounded list of snapshot candidate lines, and a partial ref frame generation. +Thus, the caller can immediately retry one listed candidate. +On the direct XCTest path, this applies only to mutating dispatches. +They count all raw exact matches before hittability can select a winner, and they delegate multiple matches to the runtime classifier. +Reads (`querySelector`, and so `get`/`is`/`wait`) keep the prior rule: + +- Prefer the single hittable match. +- Report ambiguity only if hittable matches compete. + +The reason: a read has no side effect to guard, and a fail-closed read would turn a decorative duplicate into an error. +Maestro's explicit expected-point/non-hittable compatibility path remains intentionally separate. ### 2026-09-11 amendment: one collapse for one control, at both doors -Two clauses of the amendment above went stale as the read paths moved, and the -structural rule never reached the read door. - -- `querySelector` no longer backs `get`/`is`/`wait`. Those reads resolve against a - capture, through the `readUnique`/`readAny`/`readText` rows of - `packages/selectors/src/selector-pipeline.ts`, so the runner's hittable-preference - rule governs only the direct XCTest paths that still query it: the direct-iOS - touch fast path and the off-screen target probe. -- A control reported through its own accessibility wrapper answers a selector - twice. A regular iOS snapshot omits unverified hittability, so the ladder that - relates a wrapper to its control cannot fire, and #2482 collapsed that chain for - mutating resolution only: `is visible` and `get attrs` refused the same screen - as ambiguous while `press` tapped it. The collapse now sits beside the - classification that asks for it (`resolveUnverifiedWrapperControl`) and applies - where a refusal was the answer: the uniqueness rows — `is `, - `get attrs`, `screenshot --crop-on` — resolve the control instead of reporting - no match. A row that resolves before any refusal is untouched by it, and a - wrapper chain always resolves before one: depth separates a wrapper from its - control, so `get text` ranks onto the control and `wait`/`is exists` answer from - the document-order head without asking which element was meant. A candidate set - the rule does not recognize as one control — a cell and the button inside it, or - matches in distinct subtrees — still refuses. -- Replay verifies a recorded target by resolving its recorded selector again under - the same row's refusal rules, so the screen dispatch had resolved read as an - identity mismatch on the step's first replay. Verification names the collapsed - control too, which is the node dispatch acted on and the node the recorded - identity carries. +Two clauses of the amendment above became stale when the read paths moved. +Also, the structural rule never reached the read door. + +- `querySelector` no longer backs `get`/`is`/`wait`. + Those reads resolve against a capture, through the `readUnique`/`readAny`/`readText` rows of `packages/selectors/src/selector-pipeline.ts`. + Thus, the runner's hittable-preference rule governs only the direct XCTest paths that still query it: the direct-iOS touch fast path and the off-screen target probe. +- A control reported through its own accessibility wrapper answers a selector twice. + A regular iOS snapshot omits unverified hittability, so the ladder that relates a wrapper to its control cannot fire. + #2482 collapsed that chain for mutating resolution only. + As a result, `is visible` and `get attrs` refused the same screen as ambiguous, while `press` tapped it. + The collapse now sits beside the classification that asks for it (`resolveUnverifiedWrapperControl`). + The collapse applies where a refusal was the answer. + The uniqueness rows resolve the control instead of reporting no match: + - `is `. + - `get attrs`. + - `screenshot --crop-on`. + The collapse does not touch a row that resolves before any refusal. + A wrapper chain always resolves before a refusal, because depth separates a wrapper from its control. + Thus, `get text` ranks onto the control. + `wait`/`is exists` answer from the document-order head, and do not ask which element was meant. + A candidate set that the rule does not recognize as one control still refuses. + A cell and the button inside it, or matches in distinct subtrees, are such candidate sets. +- Replay verifies a recorded target: it resolves the recorded selector again under the same row's refusal rules. + Thus, the screen that dispatch had resolved read as an identity mismatch on the first replay of the step. + Verification names the collapsed control too. + That control is the node that dispatch acted on, and the node that the recorded identity carries. ### Synthesized iOS gesture policy -Synthesized iOS gestures (`scroll`, synthesized coordinate `tap`, synthesized -`drag`, and synthesized `sequence` tap/drag steps) are intentionally not folded -into the element dispatch-path matrix above. They do not resolve selectors or -refs, so claiming the element-targeting guarantees would be misleading. Their -AX-health, keyboard-probe, frame-source, activation-preflight, and -XCTest-coordinate fallback rules stay runner-local in -`RunnerTests+SynthesizedGesturePolicy.swift`. That Swift policy is the source of -truth because the current table has only three behaviors: - -- coordinate synthesized tap, standalone, as a `sequence` step, or as the - in-app `back` top-leading tap, never probes keyboards and may use the - coordinate fallback; -- default iOS scroll probes keyboards only after AX is known healthy and must - not fall back to `XCUICoordinate`; -- synthesized one-contact `gesture` plans may still use the coordinate fallback - before AX health is known, but stop using it once a snapshot stamps AX - unavailable. - -The non-obvious parts are covered by gated XCTest policy tests instead of a -cross-language mirror. A future sibling registry should only be introduced once -gesture policy grows beyond this small runner-local table or needs host-side -tooling to consume it directly. Synthesized coordinate contexts use screenshot -dimensions as their only frame source; cheaper frame sources such as -`app.frame`, windows, accessibility frames, or native screen bounds are -intentionally excluded because they can diverge from full-screen screenshot -coordinates on affected simulators. - -Target-authored `gesture drag` is classified as the `target-drag` path above because it resolves two -elements and therefore inherits the element guarantees and ADR 0012 identity requirements for both. +The element dispatch-path matrix above intentionally does not include synthesized iOS gestures: + +- `scroll`. +- Synthesized coordinate `tap`. +- Synthesized `drag`. +- Synthesized `sequence` tap/drag steps. + +They do not resolve selectors or refs, so a claim of the element-targeting guarantees would be misleading. +Their AX-health, keyboard-probe, frame-source, activation-preflight, and XCTest-coordinate fallback rules stay runner-local in `RunnerTests+SynthesizedGesturePolicy.swift`. +That Swift policy is the source of truth, because the current table has only three behaviors: + +- A coordinate synthesized tap never probes keyboards and may use the coordinate fallback. + This applies to a standalone tap, a `sequence` step, and the in-app `back` top-leading tap. +- Default iOS scroll probes keyboards only after AX is known healthy. + It must not fall back to `XCUICoordinate`. +- Synthesized one-contact `gesture` plans may still use the coordinate fallback before AX health is known. + After a snapshot stamps AX unavailable, they stop using it. + +Gated XCTest policy tests cover the non-obvious parts, instead of a cross-language mirror. +A future sibling registry should be introduced only once one of these conditions is true: + +- Gesture policy grows beyond this small runner-local table. +- Host-side tooling needs to consume gesture policy directly. + +Synthesized coordinate contexts use screenshot dimensions as their only frame source. +They intentionally exclude cheaper frame sources, such as: + +- `app.frame`. +- Windows. +- Accessibility frames. +- Native screen bounds. + +The reason: those sources can diverge from full-screen screenshot coordinates on affected simulators. + +Target-authored `gesture drag` is classified as the `target-drag` path above, because it resolves two elements. +Therefore, it inherits the element guarantees and the ADR 0012 identity requirements for both elements. Coordinate-authored drag remains outside this matrix. -Two-contact pan/pinch/rotate/transform planning is now owned by the typed -gesture-plan contract in [ADR 0013](0013-unified-gesture-plans.md). It remains a -sibling of this element-targeting matrix: coordinate gestures do not acquire -selector/ref guarantees merely because their native executor uses XCTest. +The typed gesture-plan contract in [ADR 0013](0013-unified-gesture-plans.md) now owns two-contact pan/pinch/rotate/transform planning. +That contract remains a sibling of this element-targeting matrix. +A coordinate gesture does not acquire selector/ref guarantees only because its native executor uses XCTest. ## Migration plan -Each step lands green and independently useful: +Each step lands green and is independently useful. +Steps 1, 2, 4, and 5 are complete, and git history keeps them. +Step 3 is not complete: -1. **Registry + gate** with an honest initial classification (waivers linked - to issues). No behavior change. -2. **Response builder consolidation** (`buildInteractionResponseData`) + the - hand-rolled-literal guard test. -3. **Golden fixture tables** + `TapPointPolicy` Swift extraction + the three - consumers (vitest, XCTest, fake runner). -4. **Contract scenario suite** + registry-driven coverage gate; port the - Bluesky fixtures from #1075's tests into the permanent tables. -5. **Descriptor timeout policy**; delete the two client-side command lists. +3. **Golden fixture tables** + `TapPointPolicy` Swift extraction + the three consumers (vitest, XCTest, fake runner). ## Alternatives considered -- **Delete the fast paths** (single resolution spine): unacceptable — the - direct iOS path saves a full snapshot round trip per interaction and the - ref fast path is the backbone of replay throughput. The problem is not that - fast paths exist; it is that their trade-offs were implicit. -- **Typestate/branded types** ("cannot construct a response without proof the - guards ran"): cannot reach across the wire into Swift, and the proof tokens - go viral through every signature for marginal gain over the - registry-plus-gate split. Types are used where they are strongest — - completeness of the declaration — and tests where they are strongest — - truth of the declaration. -- **More integration tests without the registry**: this is the status quo - plus effort. Without the matrix as code, nothing forces a new path to - acquire the existing suite, which is exactly how this week's bugs happened. +- **Delete the fast paths** (single resolution spine): unacceptable. + The direct iOS path saves a full snapshot round trip per interaction. + The ref fast path is the backbone of replay throughput. + The problem is not that fast paths exist. + The problem is that their trade-offs were implicit. +- **Typestate/branded types** ("cannot construct a response without proof the guards ran"): this cannot reach across the wire into Swift. + Also, the proof tokens would spread through every signature, for a marginal gain over the registry-plus-gate split. + Types are used where they are strongest: completeness of the declaration. + Tests are used where they are strongest: truth of the declaration. +- **More integration tests without the registry**: this is the status quo plus effort. + Without the matrix as code, nothing forces a new path to acquire the existing suite. + That is exactly how this week's bugs happened. ### Optional observation before an iOS coordinate tap -A coordinate tap must not depend on a preceding XCTest snapshot failure. Its -optional text-input lookup may establish a concrete identity for a later bare -`type`; an absent or unavailable lookup establishes no typing witness. A runner -snapshot penalty can skip this work, but is only a performance optimization. +A coordinate tap must not depend on a preceding XCTest snapshot failure. +Its optional text-input lookup may establish a concrete identity for a later bare `type`. +An absent or unavailable lookup establishes no typing witness. +A runner snapshot penalty can skip this work, but the penalty is only a performance optimization. + +The lookup owns a thread-bound issue scope in the runner recorder and returns a typed result. +Any recorded issue discards partial candidates, including an issue that AX suppression otherwise handles. +The scope excludes gesture dispatch and required text-entry reads. +Those failures retain the existing mutation-outcome rules. + +The iOS PR lane exercises: -The lookup owns a thread-bound issue scope in the runner recorder and returns a -typed result. Any recorded issue, including one otherwise handled by AX suppression, -discards partial candidates. The scope excludes gesture dispatch and required -text-entry reads. Those failures retain the existing mutation-outcome rules. +- A fresh runner with an unavailable probe. +- A suppressed AX issue with a matching candidate. +- A healthy coordinate tap followed by typing. +- Failures outside the optional observation scope. -The iOS PR lane exercises a fresh runner with an unavailable probe, a suppressed -AX issue with a matching candidate, healthy coordinate tap followed by typing, -and failures outside the optional observation scope. These tests must not seed a -snapshot penalty to make the first tap safe. +These tests must not seed a snapshot penalty to make the first tap safe. -The recorder consumes optional-read issues before forwarding to XCTest. XCTest's -expected-failure API must not own this scope: in a long-lived command test it can -complete the enclosing test even when the command response succeeds. +The recorder consumes optional-read issues before it forwards issues to XCTest. +XCTest's expected-failure API must not own this scope. +The reason: in a long-lived command test, that API can complete the enclosing test even if the command response succeeds. diff --git a/docs/adr/0013-unified-gesture-plans.md b/docs/adr/0013-unified-gesture-plans.md index 590521a7d2..1566af8feb 100644 --- a/docs/adr/0013-unified-gesture-plans.md +++ b/docs/adr/0013-unified-gesture-plans.md @@ -4,215 +4,272 @@ Accepted +Completed process history is in git history. + +## Rules at a glance + +- Normalize public gesture input once in `packages/contracts/src/gesture-normalization.ts`. +- Plan canonical intent in `packages/contracts/src/gesture-plan.ts`. Keep contact topology separate from motion. +- Reject removed deprecated arguments with actionable `INVALID_ARGS` messages. Do not silently reinterpret them. +- Carry Maestro's `endpoint-hold` profile only on the private daemon wire. A genuine public `pan` uses `timed-pan`. +- Every injected sample must fit the freshly resolved active-app interaction viewport. + Otherwise, the request fails before injection with `GESTURE_TRAJECTORY_OUT_OF_BOUNDS`. + The planner does not clamp points, cache the viewport, or distort the requested components. +- Android touch execution never falls back to `adb input swipe`. +- On Android, gestures do not stop or restart the snapshot session. +- Keep the positional `.ad` codec. One `PUBLIC_GESTURE_SYNTAX` table defines argument arity for the CLI argv parse and the `.ad` line parse. +- For repeated coordinate swipes, the combined planned gesture and pause schedule must fit within 60 seconds. +- For target-authored drag, resolve both endpoints before device injection. On replay, verify both identities before pointer-down. +- Admit target-authored drag only on Android touch devices and iOS/iPadOS. +- iOS timing changes require live simulator evidence from an independent app-observed postcondition. + ## Context -Gesture intent was previously interpreted at several private boundaries: command aliases produced -positional strings, daemon dispatch reparsed them, the `Interactor` exposed parallel semantic -methods, and each platform derived its own two-contact geometry. That duplicated validation, -responses, and behavior behind the three public surfaces: CLI, Node.js, and MCP. +Several private boundaries previously interpreted gesture intent: + +- Command aliases produced positional strings. +- Daemon dispatch parsed them again. +- The `Interactor` exposed parallel semantic methods. +- Each platform derived its own two-contact geometry. -Two-contact geometry also differed by platform. Android used a fixed radius while Apple derived a -radius from the app frame. Neither path validated every planned point against the active -interaction viewport before injection. +This duplicated validation, responses, and behavior behind the three public surfaces: CLI, Node.js, and MCP. -`scroll` remains a separate command because it owns viewport/edge traversal and content-state -verification rather than a single physical gesture. Its Android physical movement still lowers to -the Android planned-touch executor. +Two-contact geometry also differed by platform. +Android used a fixed radius, while Apple derived a radius from the app frame. +Neither path validated every planned point against the active interaction viewport before injection. + +`scroll` remains a separate command because it owns viewport/edge traversal and content-state verification, not a single physical gesture. +Its Android physical movement still lowers to the Android planned-touch executor. ## Decision -Public gesture inputs normalize once in `packages/contracts/src/gesture-normalization.ts`. This is the -explicit public compatibility boundary: canonical semantic intent is produced before entering the -runtime. Deprecated arguments that have been removed (timed `swipe`, timed `gesture fling`, -`gesture rotate` `velocity`) are rejected with actionable `INVALID_ARGS` messages rather than -silently reinterpreted. The private daemon wire remains free to carry compatibility-only hints -that do not appear on public surfaces; Maestro timed swipes use -`internal.gestureExecutionProfile: 'endpoint-hold'` to preserve iOS fast-swipe-then-hold behavior -while still routing through the canonical `pan` input. - -The runtime plans canonical intent in `packages/contracts/src/gesture-plan.ts`. Contact topology is separate -from motion: - -- one contact: pan, fling, or target-authored drag with a complete pointer trajectory and an explicit - execution profile; -- two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories. - -`swipe` without a duration remains public sugar for a fixed-duration fling. Timed public forms -(`swipe x1 y1 x2 y2 durationMs`, `gesture fling direction x y distance durationMs`, -`gesture swipe preset durationMs`, and `gesture rotate degrees x y velocity`) are rejected; callers -must use `gesture pan` for deliberate timed translation and `gesture rotate` without `velocity`. -Maestro-authored swipes normalize to the canonical `pan` input (origin, delta, durationMs) and carry -the `endpoint-hold` execution profile through a daemon-internal compatibility seam -(`internal.gestureExecutionProfile`). This preserves the iOS fast-swipe-then-hold behavior that -matches Maestro's XCTest driver, without exposing the profile on the public command surface. Maestro -materializes its 400 ms default when duration is omitted. A genuine public `pan` uses the `timed-pan` -profile, so compatibility-only paths retain their release behavior without becoming a new semantic -intent. Pinch fixes translation and rotation at zero; rotate fixes -translation at zero and scale at one; two-finger pan fixes scale at one and rotation at zero; -transform can apply all three -components atomically. Intent remains on the plan even when aliases share an executor. - -The planner owns deterministic multi-touch geometry. Contacts start at -90 degrees, except Android -pinch starts horizontally because a vertical pinch is captured by common vertical app scroll -containers before the pinch recognizer activates. The same explicit planning profile preserves the -proven frame-count convention for dense two-contact trajectories: Android rounds while Apple -truncates the duration/16 ms frame count. The Android transport lowerer uses that same Android -sampling profile for one-contact endpoint plans. These are planner inputs, not adapter-generated -trajectories. The larger of pinch's initial and final spans is 40% of the -viewport's shorter side, preserving the proven Apple pinch geometry; other two-contact intents use -25% to keep translation and rotation trajectories compact. The other span follows from the requested scale, and both must -satisfy a 48-point reliability floor. Combined transforms progress translation, scale, and rotation -together inside one uninterrupted two-contact sequence so recognizers observe every intent without an -adapter regenerating geometry. The planner does not clamp points, cache the viewport, or distort -the requested components. Every injected sample must fit the freshly resolved active-app -interaction viewport; otherwise the request fails before injection with -`GESTURE_TRAJECTORY_OUT_OF_BOUNDS` and actionable details. Span and angle remain internal because -no established automation use case justifies a public tuning surface. +Public gesture inputs normalize once in `packages/contracts/src/gesture-normalization.ts`. +This is the explicit public compatibility boundary. +Canonical semantic intent is produced before the input enters the runtime. +These removed deprecated arguments are rejected with actionable `INVALID_ARGS` messages, not silently reinterpreted: + +- Timed `swipe`. +- Timed `gesture fling`. +- `gesture rotate` `velocity`. + +The private daemon wire remains free to carry compatibility-only hints that do not appear on public surfaces. +Maestro timed swipes use `internal.gestureExecutionProfile: 'endpoint-hold'` and still route through the canonical `pan` input. +This preserves the iOS fast-swipe-then-hold behavior. + +The runtime plans canonical intent in `packages/contracts/src/gesture-plan.ts`. +Contact topology is separate from motion: + +- One contact: pan, fling, or target-authored drag with a complete pointer trajectory and an explicit execution profile. +- Two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories. + +Public `swipe` without a duration remains an alias for a fixed-duration fling. +These timed public forms are rejected: + +- `swipe x1 y1 x2 y2 durationMs`. +- `gesture fling direction x y distance durationMs`. +- `gesture swipe preset durationMs`. +- `gesture rotate degrees x y velocity`. + +For deliberate timed translation, callers must use `gesture pan`. +For rotation, callers must use `gesture rotate` without `velocity`. +Maestro-authored swipes normalize to the canonical `pan` input (origin, delta, durationMs). +They carry the `endpoint-hold` execution profile through a daemon-internal compatibility seam (`internal.gestureExecutionProfile`). +This preserves the iOS fast-swipe-then-hold behavior that matches Maestro's XCTest driver. +The public command surface does not expose the profile. +If the duration is omitted, Maestro materializes its 400 ms default. +A genuine public `pan` uses the `timed-pan` profile. +Thus, compatibility-only paths retain their release behavior and do not become a new semantic intent. + +Each intent fixes some plan components: + +- Pinch fixes translation and rotation at zero. +- Rotate fixes translation at zero and scale at one. +- Two-finger pan fixes scale at one and rotation at zero. +- Transform can apply all three components atomically. + +Intent remains on the plan even when aliases share an executor. + +The planner owns deterministic multi-touch geometry. +Contacts start at -90 degrees, except that Android pinch starts horizontally. +The reason is that common vertical app scroll containers capture a vertical pinch before the pinch recognizer activates. +The same explicit planning profile preserves the proven frame-count convention for dense two-contact trajectories. +Android rounds the duration/16 ms frame count. +Apple truncates it. +The Android transport lowerer uses that same Android sampling profile for one-contact endpoint plans. +These are planner inputs, not adapter-generated trajectories. + +For pinch, the larger of the initial and final spans is 40% of the viewport's shorter side. +This preserves the proven Apple pinch geometry. +Other two-contact intents use 25% to keep translation and rotation trajectories compact. +The requested scale determines the other span. +Both spans must satisfy a 48-point reliability floor. +Combined transforms progress translation, scale, and rotation together inside one uninterrupted two-contact sequence. +Thus, recognizers observe every intent, and no adapter regenerates geometry. +The planner does not clamp points, cache the viewport, or distort the requested components. +Every injected sample must fit the freshly resolved active-app interaction viewport. +Otherwise, the request fails before injection with `GESTURE_TRAJECTORY_OUT_OF_BOUNDS` and actionable details. +Span and angle remain internal because no established automation use case justifies a public tuning surface. Platform adapters consume the canonical plan: -- Android's `executeAndroidTouchPlan` adapter seam sends planned touch, including gesture plans plus - the physical movement for scroll and long-press, to provider-native touch injection when - available, otherwise to the bundled instrumentation helper. One-contact endpoint plans lower in - `packages/platform-android/src/touch-plan-lowering.ts` to approximately 16 ms transport samples before - either injection path. Controlled directional scrolls accelerate for one frame, then decelerate - through release within the requested duration, without an appended tail. Inertial scrolls and - general one-contact plans retain linear interpolation; two-contact plans retain their exact - planned samples. Easing reduces release momentum but does not guarantee an exact content offset. Transport samples are typed as - strictly denser than the canonical endpoint pair, so skipping that lowering is a type error at - the injection seams instead of a silently sparse gesture. A stationary long-press needs no - viewport on the helper path; the executor adds the paired provider-owned viewport only for - provider-native touch. Android touch execution never falls back to `adb input swipe`. Public - scroll durations below one 16 ms planner frame normalize to that physical minimum and report the - executed duration. Scroll evidence reports absolute injected coordinates against zero-origin - extents that include the viewport offset. Because Android permits only one instrumentation owner - of `UiAutomation`, snapshot capture, gesture viewport resolution, and planned-touch injection - share one bundled automation helper: a live persistent helper session executes touch commands - directly, and without one the same helper runs one-shot. Nothing stops the snapshot session - around gestures anymore (amended 2026-07, issue #1275; previously a separate one-shot - multi-touch helper forced a session stop/restart around every local gesture). -- iOS lowers one-contact endpoint-hold plans to the established fast-swipe synthesis profile. That - profile reaches the endpoint in 100 ms, then holds there for the planned duration before lifting, - matching Maestro's XCTest driver. One-contact plans are linear and therefore carry only their - start and end samples, with the authored duration between them. Two-contact plans convert every - planned point to native orientation and feed the exact arrays to the private XCTest event bridge. - macOS lowers a one-contact plan to its drag executor and tvOS lowers it to remote direction. Core - admission and the Apple adapter both consume the same shared multi-touch support policy; - multi-touch remains capability-gated to iOS simulators. - - iOS deliberately retains three one-contact motion schedules behind the shared private-XCTest event - bridge. Apple's public drag API models the phases independently as an - [initial hold, movement velocity, and destination hold](https://developer.apple.com/documentation/xcuiautomation/xcuicoordinate/press%28forduration%3Athendragto%3Awithvelocity%3Athenholdforduration%3A%29). - UIKit likewise reports the velocity at finger-up together with the scroll view's predicted - [resting content offset](https://developer.apple.com/documentation/uikit/uiscrollviewdelegate/scrollviewwillenddragging%28_%3Awithvelocity%3Atargetcontentoffset%3A%29), - then applies the scroll view's configured - [post-lift deceleration rate](https://developer.apple.com/documentation/uikit/uiscrollview/decelerationrate-swift.property). - The distribution of movement over time is therefore observable gesture input, not an adapter - implementation detail. - - The three schedules shape that velocity differently. `endpoint-hold` moves quickly for 100 ms and - becomes stationary before lift. `timed-pan` and target-authored drag submit the authored samples - unchanged, preserving piecewise-linear movement plus explicit source and destination holds. The - runner's fused `scroll` reuses `endpoint-hold` for inertial release; controlled release instead - expands the movement to evenly spaced samples, one per roughly 16 ms capped at 30, using cubic - ease-out `s(t) = 1 - (1 - t)³`. A linear segment has constant movement velocity through its - endpoint unless a destination hold follows it; cubic ease-out decelerates from the first movement - and has zero slope at lift, with a starting velocity three times its average. Identical endpoints - and total durations can consequently produce different recognizer and deceleration outcomes. - Neither a destination hold nor an analytically zero endpoint slope proves a controlled release by - itself: XCTest event sampling and app recognizer thresholds can still leave observable post-lift - motion, so live evidence must measure the resulting content offset after pointer-up. - - Live iOS characterization of an eased sampled schedule against the linear ones in - [issue #1586](https://github.com/callstack/agent-device/issues/1586) confirmed that distinction: - the schedules crossed the same fling-recognizer thresholds in the tested range but produced - materially different post-release ScrollView positions and long-duration recognition behavior. The - distinction is intentional policy at the Apple adapter boundary, not a second interpretation of a - `GesturePlan`; changes require live evidence for both recognizer activation and post-release - content movement. -- WebDriver lowers a supported plan to synchronized W3C pointer action sources. A one-contact - endpoint plan becomes pointer down, one timed W3C `pointerMove` from start to end, and pointer up; - the driver owns interpolation across that W3C tick. Multi-touch remains capability-gated until a - provider proves it. - -The `Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of a -method per semantic alias. The old scalar Apple and Android multi-touch executors and the -public-command alias-to-positionals-to-reparse route are deleted. `.ad` keeps its established -positional syntax through one named codec; CLI, Node.js, and MCP send structured input. Providers -should compose transport/device bindings with the shared platform adapter rather than reimplement -the interaction runtime. - -That `.ad` codec is the script format's own syntax, not a compatibility shim, and is not scheduled -for removal (amended 2026-07, issue #1216). Its only remaining callers are the CLI argv parse and -the `.ad` line parse — both the current public syntax — so there is nothing left to migrate off. -A structured `.ad` gesture payload was rejected: it would make recordings unreadable and ungreppable -for no behavioral gain. - -Argument arity for both callers comes from one table (`PUBLIC_GESTURE_SYNTAX`), so a form removed -from the CLI is removed from `.ad` in the same edit. A `.ad` script that still carries a removed -positional fails when the script is parsed, before the replay executes any device action, with the -offending line and its rewrite. Arity is the only thing checked at parse time: `${VAR}` tokens -resolve after planning, and interpolation never splits a token, so the count is decidable while the -values are not. The removal process these inputs follow — announce, warn for one minor release, -publish the migration guide, prove the repository is clean, then remove the branch and its tests — -is documented in the public -[gesture migration guide](https://oss.callstack.com/agent-device/docs/migrating-gestures). - -Repeated coordinate swipes are bounded at the public command contract and daemon trust boundary. -Individual count and pause limits prevent pathological fields, while the combined planned gesture -and pause schedule must fit within 60 seconds so valid fields cannot compose into an unbounded -session lock. - -Public two-finger pan is additive: `pointerCount?: 1 | 2` on pan and CLI -`--pointer-count 2`; omission remains one contact. Responses share the canonical -`kind`, `durationMs`, `pointerCount`, `from`, and `to` fields, followed by backend evidence. +- **Android.** The `executeAndroidTouchPlan` adapter seam sends planned touch. + Planned touch includes gesture plans and the physical movement for scroll and long-press. + If provider-native touch injection is available, the seam sends planned touch to it. + Otherwise, the seam sends planned touch to the bundled instrumentation helper. + `packages/platform-android/src/touch-plan-lowering.ts` lowers one-contact endpoint plans to approximately 16 ms transport samples before either injection path. + Controlled directional scrolls accelerate for one frame, then decelerate through release within the requested duration. + They do not append a tail. + Inertial scrolls and general one-contact plans retain linear interpolation. + Two-contact plans retain their exact planned samples. + Easing reduces release momentum but does not guarantee an exact content offset. + Transport samples are typed as strictly denser than the canonical endpoint pair. + Thus, a skipped lowering is a type error at the injection seams, not a silently sparse gesture. + A stationary long-press needs no viewport on the helper path. + The executor adds the paired provider-owned viewport only for provider-native touch. + Android touch execution never falls back to `adb input swipe`. + Public scroll durations below one 16 ms planner frame normalize to that physical minimum and report the executed duration. + Scroll evidence reports absolute injected coordinates against zero-origin extents that include the viewport offset. + Android permits only one instrumentation owner of `UiAutomation`. + Snapshot capture, gesture viewport resolution, and planned-touch injection therefore share one bundled automation helper. + A live persistent helper session executes touch commands directly. + Without a live session, the same helper runs one-shot. + Gestures do not stop or restart the snapshot session (amended 2026-07, issue #1275). +- **Apple.** iOS lowers one-contact endpoint-hold plans to the established fast-swipe synthesis profile. + That profile reaches the endpoint in 100 ms, then holds there for the planned duration before it lifts. + This matches Maestro's XCTest driver. + One-contact plans are linear and therefore carry only their start and end samples, with the authored duration between them. + Two-contact plans convert every planned point to native orientation and send the exact arrays to the private XCTest event bridge. + macOS lowers a one-contact plan to its drag executor. + tvOS lowers a one-contact plan to remote direction. + Core admission and the Apple adapter both consume the same shared multi-touch support policy. + Multi-touch remains capability-gated to iOS simulators. + + iOS deliberately retains three one-contact motion schedules behind the shared private-XCTest event bridge. + Apple's public drag API models the phases independently as an [initial hold, movement velocity, and destination hold](https://developer.apple.com/documentation/xcuiautomation/xcuicoordinate/press%28forduration%3Athendragto%3Awithvelocity%3Athenholdforduration%3A%29). + UIKit also reports the finger-up velocity together with the scroll view's predicted [resting content offset](https://developer.apple.com/documentation/uikit/uiscrollviewdelegate/scrollviewwillenddragging%28_%3Awithvelocity%3Atargetcontentoffset%3A%29). + UIKit then applies the scroll view's configured [post-lift deceleration rate](https://developer.apple.com/documentation/uikit/uiscrollview/decelerationrate-swift.property). + The distribution of movement over time is therefore observable gesture input, not an adapter implementation detail. + + The three schedules shape that velocity differently: + + - `endpoint-hold` moves quickly for 100 ms and becomes stationary before lift. + - `timed-pan` and target-authored drag submit the authored samples unchanged. + They preserve piecewise-linear movement plus explicit source and destination holds. + - The runner's fused `scroll` reuses `endpoint-hold` for inertial release. + For controlled release, the fused `scroll` instead expands the movement to evenly spaced samples, one per approximately 16 ms, capped at 30. + Controlled release uses cubic ease-out `s(t) = 1 - (1 - t)³`. + + A linear segment has constant movement velocity through its endpoint, unless a destination hold follows it. + Cubic ease-out decelerates from the first movement and has zero slope at lift. + Its starting velocity is three times its average. + Thus, identical endpoints and total durations can produce different recognizer and deceleration outcomes. + Neither a destination hold nor an analytically zero endpoint slope proves a controlled release by itself. + XCTest event sampling and app recognizer thresholds can still leave observable post-lift motion. + Live evidence must therefore measure the resulting content offset after pointer-up. + + Live iOS characterization in [issue #1586](https://github.com/callstack/agent-device/issues/1586) compared an eased sampled schedule with the linear schedules. + It confirmed that distinction. + In the tested range, the schedules crossed the same fling-recognizer thresholds. + However, they produced materially different post-release ScrollView positions and long-duration recognition behavior. + This distinction is intentional policy at the Apple adapter boundary, not a second interpretation of a `GesturePlan`. + Changes require live evidence for both recognizer activation and post-release content movement. +- **WebDriver.** WebDriver lowers a supported plan to synchronized W3C pointer action sources. + A one-contact endpoint plan becomes pointer down, one timed W3C `pointerMove` from start to end, and pointer up. + The driver owns interpolation across that W3C tick. + Multi-touch remains capability-gated until a provider proves it. + +The `Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of one method per semantic alias. +The old scalar Apple and Android multi-touch executors are deleted. +The public-command alias-to-positionals-to-reparse route is also deleted. +`.ad` keeps its established positional syntax through one named codec. +CLI, Node.js, and MCP send structured input. +Providers should compose transport/device bindings with the shared platform adapter instead of a reimplementation of the interaction runtime. + +That `.ad` codec is the script format's own syntax, not a compatibility shim. +It is not scheduled for removal (amended 2026-07, issue #1216). +Its only remaining callers are the CLI argv parse and the `.ad` line parse. +Both are the current public syntax, so nothing is left to migrate off. +A structured `.ad` gesture payload was rejected. +It would make recordings unreadable and impossible to search with grep, for no behavioral gain. + +One table, `PUBLIC_GESTURE_SYNTAX`, defines argument arity for both callers. +Thus, a form removed from the CLI is removed from `.ad` in the same edit. +If a `.ad` script still carries a removed positional, the script fails when it is parsed. +This failure occurs before the replay executes any device action. +The error names the offending line and its rewrite. +Arity is the only thing checked at parse time. +`${VAR}` tokens resolve after planning, and interpolation never splits a token. +Thus, the count is decidable at parse time, but the values are not. +These inputs follow a removal process. +The public [gesture migration guide](https://oss.callstack.com/agent-device/docs/migrating-gestures) documents it: + +1. Announce the removal. +2. Warn for one minor release. +3. Publish the migration guide. +4. Prove that the repository is clean. +5. Remove the branch and its tests. + +Repeated coordinate swipes are bounded at the public command contract and at the daemon trust boundary. +Individual count and pause limits prevent pathological fields. +The combined planned gesture and pause schedule must fit within 60 seconds. +Thus, valid fields cannot compose into an unbounded session lock. + +Public two-finger pan is additive: `pointerCount?: 1 | 2` on pan, and CLI `--pointer-count 2`. +Omission remains one contact. +Responses share the canonical `kind`, `durationMs`, `pointerCount`, `from`, and `to` fields, followed by backend evidence. Recording/replay keeps its existing public command identity and session semantics. -Target-authored drag is additive: `gesture drag ` accepts a selector or pinned -snapshot ref at each endpoint, resolves both endpoints before device injection, then lowers their center -points to one uninterrupted single-pointer plan. The plan holds at the source, moves over the authored -duration, optionally holds at the destination, and releases. Both endpoint-resolution disclosures and -portable selector chains are returned to the caller. Recordings replace session-local refs at both -endpoints with those selector chains and attach one `targets-v1` annotation containing independent source -and destination identity evidence. Replay verifies both identities before pointer-down and guards both -independent endpoint resolutions against the verified elements. Ref admission happens for both endpoints -before either is dispatched, and the usual mutation boundary expires the frame after the gesture. Because -this contract includes the source hold, timed movement, and destination hold phases, target-authored drag -is admitted only on Android touch devices and iOS/iPadOS, whose adapters preserve the full plan. - -ADR 0011's coordinate path remains unchanged: coordinate-authored gestures do not resolve selectors or -refs and therefore cannot claim element-targeting guarantees. Target-authored drag explicitly runs the -shared selector/ref resolution preflight before it enters the coordinate gesture executor; its endpoint -resolution and recording contracts are tested at that composition seam. +Target-authored drag is additive: `gesture drag `. +Each endpoint accepts a selector or a pinned snapshot ref. +Both endpoints resolve before device injection. +Their center points then lower to one uninterrupted single-pointer plan: + +1. Hold at the source. +2. Move over the authored duration. +3. Optionally, hold at the destination. +4. Release. + +Both endpoint-resolution disclosures and portable selector chains are returned to the caller. +Recordings replace session-local refs at both endpoints with those selector chains. +Recordings also attach one `targets-v1` annotation that contains independent source and destination identity evidence. +Before pointer-down, replay verifies both identities. +Replay also guards both independent endpoint resolutions against the verified elements. +Ref admission happens for both endpoints before either endpoint is dispatched. +After the gesture, the usual mutation boundary expires the frame. +This contract includes the source hold, timed movement, and destination hold phases. +Thus, target-authored drag is admitted only on Android touch devices and iOS/iPadOS, whose adapters preserve the full plan. + +ADR 0011's coordinate path remains unchanged. +Coordinate-authored gestures do not resolve selectors or refs and therefore cannot claim element-targeting guarantees. +Before it enters the coordinate gesture executor, target-authored drag explicitly runs the shared selector/ref resolution preflight. +Its endpoint resolution and recording contracts are tested at that composition seam. ## Consequences - CLI, Node.js, MCP, runtime, and platform adapters share one normalization and planning model. -- Adding an ergonomic gesture alias does not add a platform implementation. -- One-finger pan remains the default and explicit two-finger pan retains pan intent. -- The active viewport is resolved for each gesture, so rotation, keyboard, and window changes do - not use stale geometry. -- On bare ADB, Android scroll and long-press require the bundled automation helper (the snapshot - helper APK) and `UiAutomation`; helper installation or runtime failure is surfaced directly - rather than degrading to an approximate `adb input swipe`. -- Canonical one-contact plans contain only two samples: a 10-second pan is two shared-plan samples - instead of 626. Android expands that plan only at the transport boundary, while two-contact - plans remain cadence-bounded because their synchronized geometry is part of their contract. -- Unit tests cover canonical plan shape, Android lowering, helper/provider payloads, and WebDriver - action construction. They cannot prove timing or event delivery inside the private XCTest bridge, - so iOS timing changes require live simulator evidence from an independent app-observed - postcondition. Runner start/end uptime deltas can supplement that evidence, but request echoes or - internal timing alone do not prove that the app recognized the intended gesture. +- A new ergonomic gesture alias does not add a platform implementation. +- One-finger pan remains the default, and explicit two-finger pan retains pan intent. +- The active viewport is resolved for each gesture. + Rotation, keyboard, and window changes therefore do not use stale geometry. +- On bare ADB, Android scroll and long-press require the bundled automation helper, which is the snapshot helper APK, and `UiAutomation`. + Helper installation or runtime failure is surfaced directly. + It does not degrade to an approximate `adb input swipe`. +- Canonical one-contact plans contain only two samples. + A 10-second pan therefore has two shared-plan samples instead of 626. + Android expands that plan only at the transport boundary. + Two-contact plans remain cadence-bounded because their synchronized geometry is part of their contract. +- Unit tests cover canonical plan shape, Android lowering, helper/provider payloads, and WebDriver action construction. + They cannot prove timing or event delivery inside the private XCTest bridge. + iOS timing changes therefore require live simulator evidence from an independent app-observed postcondition. + Runner start/end uptime deltas can supplement that evidence. + Request echoes or internal timing alone do not prove that the app recognized the intended gesture. ## Alternatives Considered -- Keep positional aliases and share only geometry math: rejected because validation, response, and - routing would still have two implementations. -- Make platform gesture APIs the source of truth: rejected because their timing, geometry, and - recognizer behavior differ and cannot provide cross-platform semantics. -- Make swipe a public synonym for pan: rejected because battle-tested gesture vocabulary treats a - fling/swipe as a quick directional throw and pan as deliberate timed translation. -- Add `two-finger-pan`: rejected because pointer count is topology, not a new motion intent. -- Expose span/angle controls: rejected until a concrete automation use case needs them. -- Consolidate scroll command semantics: rejected because edge/content verification is distinct; - only its Android physical touch execution is shared. +- **Keep positional aliases and share only geometry math.** Rejected because validation, response, and routing would still have two implementations. +- **Make platform gesture APIs the source of truth.** Rejected because their timing, geometry, and recognizer behavior differ and cannot provide cross-platform semantics. +- **Make swipe a public synonym for pan.** Rejected because established gesture vocabulary treats a fling/swipe as a quick directional throw and pan as deliberate timed translation. +- **Add `two-finger-pan`.** Rejected because pointer count is topology, not a new motion intent. +- **Expose span/angle controls.** Rejected until a concrete automation use case needs them. +- **Consolidate scroll command semantics.** Rejected because edge/content verification is distinct. + Only its Android physical touch execution is shared. diff --git a/docs/adr/0014-session-ref-frame-lifetime.md b/docs/adr/0014-session-ref-frame-lifetime.md index a8943fcf06..45d87bea33 100644 --- a/docs/adr/0014-session-ref-frame-lifetime.md +++ b/docs/adr/0014-session-ref-frame-lifetime.md @@ -2,181 +2,215 @@ ## Status -Accepted; implemented and enforced on `main` across all platforms. Fresh live evidence exercised -every production seam except Android blocking-dialog recovery, which shipped as a documented, -accepted gap: no deterministic repro exists to raise the system dialog on demand, and the seam's -transition and abort logic are pinned by fixture regressions -(`android-system-dialog-ref-frame.test.ts`, `interaction-touch-android-readiness.test.ts`). The -staged migration plan lives in this file's git history; PR #1241 was an earlier compatible -transitional fix, not this architecture migration. +Accepted. +This decision is implemented and enforced on `main` across all platforms. + +Fresh live evidence exercised every production seam except Android blocking-dialog recovery. +Android blocking-dialog recovery shipped as a documented, accepted gap, because no deterministic repro exists to raise the system dialog on demand. +Fixture regressions pin the transition and abort logic of that seam: `android-system-dialog-ref-frame.test.ts` and `interaction-touch-android-readiness.test.ts`. + +The staged migration plan is in the git history of this file. +PR #1241 was an earlier compatible transitional fix, not this architecture migration. ## Rules at a glance -Normative summary; the binding contracts and edge cases are in [Decision](#decision), and the -ref-frame vocabulary is promoted into `CONTEXT.md`. - -- A session owns at most one **ref frame** — the authorization namespace for mutation refs (epoch - exposed as `refsGeneration`, immutable source tree, `active`/`expired` state, `all` or bounded - issuance scope) — kept separate from the latest operational observation (`session.snapshot`). - The frame is one value on `SessionState`, replaced whole by a transition and never edited in - place; its type is nominal (`#`-private fields), so no module outside - `packages/daemon/src/core/ref-frame.ts` can construct a frame, edit one, or derive one from an existing - frame. What the type cannot judge is a whole frame moved unchanged — clearing the field, or - assigning another session's frame — which the field-owner gate still covers. -- A complete snapshot activates an `all` frame; `find`, settled diffs, and replay divergence - screens activate a bounded partial frame that supersedes the prior one; internal read-only - captures never activate, reindex, or expire a frame. -- Every mutating leaf expires the frame at the side-effect seam — after all pre-action guards, - immediately before the device op — with no success-only rollback; a post-dispatch failure still - leaves it expired. -- Mutation admission requires an active frame whose epoch and issuance scope authorize the ref; - rejections carry ordered typed reasons `ref_frame_expired` -> `ref_generation_mismatch` -> - `plain_ref_requires_complete_frame` -> `ref_not_issued`. -- Read-only ref consumers stay fail-open with a structured staleness warning only while the frame - retains the ref's evidence; missing evidence fails rather than resolving by positional - coincidence against a newer observation. -- Every daemon command declares a `RefFrameEffect` (`preserve` | `may-invalidate` | `delegated`) in - its descriptor daemon facet; the registry classification is completeness-gated. +This section is a normative summary. +[Decision](#decision) contains the binding contracts and edge cases. +The ref-frame vocabulary is promoted into `CONTEXT.md`. + +- A session owns at most one **ref frame**. + The ref frame is the authorization namespace for mutation refs. + It is separate from the latest operational observation (`session.snapshot`). + The frame holds an epoch exposed as `refsGeneration`, an immutable source tree, an `active`/`expired` state, and an `all` or bounded issuance scope. + The frame is one value on `SessionState`. + A transition replaces the frame whole and never edits it in place. + The frame type is nominal (`#`-private fields). + Thus, no module outside `packages/daemon/src/core/ref-frame.ts` can construct a frame, edit a frame, or derive a frame from an existing frame. + The type cannot judge a whole frame that moves unchanged, for example when code clears the field or assigns the frame of another session. + The field-owner gate still covers these moves. +- A complete snapshot activates an `all` frame. + `find`, settled diffs, and replay divergence screens activate a bounded partial frame that supersedes the prior frame. + Internal read-only captures never activate, reindex, or expire a frame. +- Every mutating leaf expires the frame at the side-effect seam: after all pre-action guards, immediately before the device operation. + There is no success-only rollback. + A post-dispatch failure still leaves the frame expired. +- Mutation admission requires an active frame whose epoch and issuance scope authorize the ref. + Rejections carry typed reasons in this order: + 1. `ref_frame_expired`. + 2. `ref_generation_mismatch`. + 3. `plain_ref_requires_complete_frame`. + 4. `ref_not_issued`. +- Read-only ref consumers stay fail-open with a structured staleness warning only while the frame retains the evidence of the ref. + Missing evidence fails, instead of resolving by positional coincidence against a newer observation. +- Every daemon command declares a `RefFrameEffect` (`preserve` | `may-invalidate` | `delegated`) in its descriptor daemon facet. + A completeness gate covers the registry classification. ## Context -Snapshot refs such as `@e12` are positional names inside one accessibility-tree namespace. They are -cheap and effective within that namespace, but they do not identify an element across navigation, -layout changes, or another capture whose positional ordering differs. A generation pin such as -`@e12~s42` identifies the namespace that issued the ref; it does not make the element stable. +Snapshot refs such as `@e12` are positional names inside one accessibility-tree namespace. +They are cheap and effective within that namespace. +But they do not identify an element across navigation, layout changes, or another capture whose positional order is different. +A generation pin such as `@e12~s42` identifies the namespace that issued the ref. +The pin does not make the element stable. + +The current session model mixes two different concepts in `session.snapshot`: -The current session model conflates two different concepts in `session.snapshot`: +1. The latest operational observation that selector matching, verification, settling, Android freshness, overlays, and diagnostics use. +2. The caller-authorized tree whose refs a later mutation may use. -1. the latest operational observation used by selector matching, verification, settling, Android - freshness, overlays, and diagnostics; and -2. the caller-authorized tree whose refs may be used for a later mutation. +This mix causes failures in both directions. +A mutation can change the device without another tree capture, so the old positional refs appear current. +Conversely, an internal read-only capture can replace `session.snapshot` and invalidate refs, even though the caller performed no mutation. +Partial outputs, such as `find`, settled diffs, and replay divergence screens, can then clear the coarse stale marker for the whole replacement tree. +They can do this even though they exposed only a few refs. -That conflation creates failures in both directions. A mutation can change the device without -capturing another tree, leaving the old positional refs apparently current. Conversely, an internal -read-only capture can replace `session.snapshot` and invalidate refs even though the caller performed -no mutation. Partial outputs such as `find`, settled diffs, and replay divergence screens can then -clear the coarse stale marker for the whole replacement tree even though they exposed only a few -refs. +Issue [#1239](https://github.com/callstack/agent-device/issues/1239) and PR [#1241](https://github.com/callstack/agent-device/pull/1241) demonstrate the first failure on iOS. +`snapshot -> press @e1 -> press @e2` can dispatch both presses when the first press performs no internal capture. +The second ref is resolved against pre-action coordinates, even if the first press navigated. +A false successful tap on the new screen is worse than a clear stale-ref failure. -Issue [#1239](https://github.com/callstack/agent-device/issues/1239) and PR -[#1241](https://github.com/callstack/agent-device/pull/1241) demonstrate the first failure on iOS: -`snapshot -> press @e1 -> press @e2` can dispatch both presses when the first press performs no -internal capture. The second ref is resolved against pre-action coordinates even if the first press -navigated. A false successful tap on the new screen is worse than a clear stale-ref failure. +The lifetime contract must meet these requirements: -The lifetime contract must be consistent across platforms and dispatch paths, preserve fast paths, -remain compatible with selector-based replay, and add no automatic capture or per-node output bytes. +- Be consistent across platforms and dispatch paths. +- Preserve fast paths. +- Remain compatible with selector-based replay. +- Add no automatic capture or per-node output bytes. ## Decision -The terms introduced below are now the implemented model; the ref-frame vocabulary is promoted into -`CONTEXT.md`. The superseded coarse `snapshotRefsStale` marker has been removed: -read-only ref staleness is now derived from frame state — a plain ref warns once the frame has expired, -and a read-only capture no longer marks refs stale because it does not expire the frame. +The terms that this section introduces are now the implemented model. +The ref-frame vocabulary is promoted into `CONTEXT.md`. +The superseded coarse `snapshotRefsStale` marker is removed. +Read-only ref staleness now derives from the frame state: + +- A plain ref warns after the frame expires. +- A read-only capture no longer marks refs stale, because it does not expire the frame. ### Ref frames are separate from operational observations -A session owns at most one **ref frame**. The frame is the authorization namespace for mutation refs -and contains: +A session owns at most one **ref frame**. +The frame is the authorization namespace for mutation refs. +It contains: + +- A seeded, monotonic **ref-frame epoch**. The existing `refsGeneration` field exposes the epoch. The existing `~s` input suffix accepts it. +- The immutable source tree that is necessary to resolve and guard the refs that the frame issued. +- A state of `active` or `expired`. +- An issuance scope of `all` or a bounded set of issued ref bodies. + +Operational captures update the latest observation that selector and diagnostic machinery use. +They do not silently replace, reactivate, or reindex the authorized ref frame. +The implementation may share immutable tree objects between the observation and the frame. +But it must not make their lifetimes the same through aliases of mutable session state. + +Only one frame is retained. +Activation of a new frame supersedes the previous frame and advances the epoch. +Historical generation-to-tree caches are not introduced. +Complete and partial frames retain the immutable source tree, because interaction guards and replay identity can depend on context outside the emitted subset. +This context includes ancestors, siblings, overlays, and the viewport. +The issuance set bounds partial authority; the evidence tree does not. +When the frame and the latest observation originate from the same capture, they share immutable capture data. +Neither transition deep-copies the tree. + +The existing `snapshotGeneration` implementation evolves behind one ref-frame module. +The `snapshotRefsStale` marker that it originally paired with is now removed. +For wire compatibility, the public name `refsGeneration` and the `@e12~s42` grammar remain unchanged. -- a seeded, monotonic **ref-frame epoch**, exposed through the existing `refsGeneration` field and - accepted through the existing `~s` input suffix; -- the immutable source tree needed to resolve and guard the refs it issued; -- a state of `active` or `expired`; and -- an issuance scope of `all` or a bounded set of issued ref bodies. +### Frame transitions -Operational captures update the latest observation used by selector and diagnostic machinery. They -do not silently replace, reactivate, or reindex the authorized ref frame. The implementation may -share immutable tree objects between the observation and frame, but it must not make their lifetimes -the same by aliasing mutable session state. +| Event | Frame transition | Plain refs | Pinned refs | +| --- | --- | --- | --- | +| Internal observation that returns no refs | None | Unchanged | Unchanged | +| Complete frame activation | New active epoch, scope `all` | Accepted from this frame | Accepted when epoch matches | +| Non-empty partial publication | New active epoch, scope = emitted refs | Rejected | Accepted only for an emitted ref at this epoch | +| First possible device-side effect | Advance once to `expired` | Rejected | Previous epoch rejected | +| Additional effects while already expired | Idempotent | Rejected | Rejected | +| Sparse, failed, or unusable capture | None | Unchanged | Unchanged | +| Session reopen | New random-seeded lifetime | Rejected until publication | Old lifetime rejected probabilistically as today | + +A complete activation normally accompanies a command result that exposes the complete ref namespace for the stored frame. +This includes an interactive or intentionally scoped snapshot. +An intentionally scoped snapshot replaces authorization with exactly its returned namespace. +It neither authorizes matching ref bodies outside that scope nor retains the old `all` scope. + +A frame-activating capture scoped by `@ref` must first admit that scope ref against the current active frame. +Scoped output reindexes refs. +To preserve the existing contract for repeated scoped snapshots after this reindex, the session retains one bounded **scope lineage**. +The lineage is for consecutive read-only repetitions of that exact scope. +The lineage stores the admitted semantic scope (label/selector chain). +It does not store the old positional tree or mutation authority. +A repeated `snapshot -s @ref` may resolve through that lineage only when the immediately preceding frame publication was the same scoped capture. +Each of these events clears the lineage: + +- The first device-side effect. +- An unrelated frame publication. +- Another scope. + +An arbitrary stale or unissued scope ref still fails closed. +It cannot borrow lineage or authorize a different subtree. +Empty, sparse, failed, or unusable repeated captures preserve the last useful frame and its still-consecutive lineage. +Selector-scoped captures follow their normal selector rules. -Only one frame is retained. Activating a new frame supersedes the previous frame and advances the -epoch; historical generation-to-tree caches are not introduced. Complete and partial frames retain -the immutable source tree because interaction guards and replay identity can depend on context outside -the emitted subset, including ancestors, siblings, overlays, and the viewport. The issuance set, not -the evidence tree, bounds partial authority. The frame and latest observation share immutable capture -data when they originate from the same capture; neither transition deep-copies the tree. +These outputs are partial external publications: -The existing `snapshotGeneration` implementation evolves behind one ref-frame module; the -`snapshotRefsStale` marker it originally paired with has since been removed. The -public name `refsGeneration` and the `@e12~s42` grammar remain unchanged for wire compatibility. +- Snapshot diffs. +- Digests that omit the namespace. +- `find`. +- Settled diffs/tails. +- Replay divergence screens. -### Frame transitions +They activate a partial frame only when at least one ref and its epoch are visible to the caller or orchestrated workflow. +An empty partial result does not supersede useful existing authority. -| Event | Frame transition | Plain refs | Pinned refs | -| ----------------------------------------- | -------------------------------------- | -------------------------- | ------------------------------------------------ | -| Internal observation that returns no refs | None | Unchanged | Unchanged | -| Complete frame activation | New active epoch, scope `all` | Accepted from this frame | Accepted when epoch matches | -| Non-empty partial publication | New active epoch, scope = emitted refs | Rejected | Accepted only for an emitted ref at this epoch | -| First possible device-side effect | Advance once to `expired` | Rejected | Previous epoch rejected | -| Additional effects while already expired | Idempotent | Rejected | Rejected | -| Sparse, failed, or unusable capture | None | Unchanged | Unchanged | -| Session reopen | New random-seeded lifetime | Rejected until publication | Old lifetime rejected probabilistically as today | - -A complete activation normally accompanies a command result that exposes the complete ref namespace -for the stored frame, including an interactive or intentionally scoped snapshot. An intentionally -scoped snapshot replaces authorization with exactly its returned namespace; it neither authorizes -matching ref bodies outside that scope nor retains the old `all` scope. - -A frame-activating capture scoped by `@ref` must first admit that scope ref against the current active -frame. To preserve the existing repeated scoped-snapshot contract after scoped output reindexes refs, -the session retains one bounded **scope lineage** for consecutive read-only repetitions of that exact -scope. The lineage stores the admitted semantic scope (label/selector chain), not the old positional -tree or mutation authority. A repeated `snapshot -s @ref` may resolve through that lineage only when -the immediately preceding frame publication was the same scoped capture. The first device-side effect, -an unrelated frame publication, or another scope clears the lineage. An arbitrary stale or unissued -scope ref still fails closed; it cannot borrow lineage or authorize a different subtree. Empty, sparse, -failed, or unusable repeated captures preserve the last useful frame and its still-consecutive lineage. -Selector-scoped captures follow their normal selector rules. +Publication of a partial frame intentionally supersedes the prior frame. +This avoids retention of an unbounded set of generation-specific trees. +It also makes the limited authority of the partial frame explicit. +Only refs named in that partial response can be used for mutation, and only with their response-level epoch. -Snapshot diffs, digests that omit the namespace, `find`, settled diffs/tails, and replay divergence -screens are partial external publications. They activate a partial frame only when at least one ref -and its epoch are visible to the caller or orchestrated workflow. An empty partial result does not -supersede useful existing authority. +An explicit, non-diff complete snapshot leaf may activate a frame inside replay or batch before the outer orchestrator returns. +This is true even when the outer response level later elides or digests that tree. +This execution-local activation is a deliberate compatibility exception for deterministic `snapshot -> @ref` scripts. +It authorizes only predeclared later bare-ref steps. +It does not: -Publishing a partial frame intentionally supersedes the prior frame. This avoids retaining an -unbounded set of generation-specific trees and makes its limited authority explicit: only refs named -in that partial response can be used for mutation, and only with their response-level epoch. +- Publish pins to the outer caller or MCP. +- Make the outer response ref-issuing. +- Add general result interpolation. -An explicit, non-diff complete snapshot leaf may activate a frame inside replay or batch before the -outer orchestrator returns, even when the outer response level later elides or digests that tree. This -execution-local activation is a deliberate compatibility exception for deterministic -`snapshot -> @ref` scripts. It authorizes only predeclared later bare-ref steps; it does not publish -pins to the outer caller or MCP, make the outer response ref-issuing, or add general result -interpolation. Internally injected, cached, freshness, and recovery captures never receive this -exception. A nested non-empty partial publication may supersede scope, but batch and replay have no -dynamic generation binding with which a later step could consume it. +Internally injected, cached, freshness, and recovery captures never receive this exception. +A nested non-empty partial publication may supersede scope. +But batch and replay have no dynamic generation binding with which a later step could consume it. ### Mutation admission and read behavior -A ref-targeting mutation is admitted only when all of the following hold: +A ref-targeting mutation is admitted only when all of these conditions hold: -1. the frame is active; -2. a pinned ref carries the current epoch, or a plain ref is used against a complete `all` frame; -3. the ref body belongs to the frame's issuance scope; and -4. normal target guards and replay target-binding verification pass. +1. The frame is active. +2. A pinned ref carries the current epoch, or a plain ref is used against a complete `all` frame. +3. The ref body belongs to the issuance scope of the frame. +4. Normal target guards and replay target-binding verification pass. -The target is resolved against the authorized frame, never against a newer operational observation -by positional coincidence. Android freshness captures may verify current identity or provide -comparison evidence, but they must not redefine which node an already-admitted `@eN` authorizes. +The target is resolved against the authorized frame, never against a newer operational observation by positional coincidence. +Android freshness captures may verify current identity or provide comparison evidence. +But they must not redefine which node an already-admitted `@eN` authorizes. -Read-only ref consumers remain fail-open with a structured staleness warning only when the retained -frame contains the evidence needed to resolve that ref. A stale read is observable and recoverable; -a stale mutation can act on the wrong element. Missing frame evidence fails rather than falling -through to a newer operational observation by positional coincidence. Reads never reactivate a frame -or convert an unissued ref into mutation authority. +Read-only ref consumers remain fail-open with a structured staleness warning only when the retained frame contains the evidence necessary to resolve that ref. +A stale read is observable and recoverable. +A stale mutation can act on the wrong element. +Missing frame evidence fails, instead of falling through to a newer operational observation by positional coincidence. +Reads never reactivate a frame or convert an unissued ref into mutation authority. -Rejected mutations use the existing error system. Admission failures are evaluated in the order -below and have typed reasons so callers can distinguish "capture a complete snapshot" from "use the -emitted pinned ref": +Rejected mutations use the existing error system. +Admission evaluates failures in this order, and each failure has a typed reason: -- `ref_frame_expired` when a side effect invalidated the current frame; -- `ref_generation_mismatch` when a pin names another epoch; -- `plain_ref_requires_complete_frame` when a plain ref targets a partial frame; and -- `ref_not_issued` when a pinned ref is outside a partial frame's issuance set. +1. `ref_frame_expired`: a side effect invalidated the current frame. +2. `ref_generation_mismatch`: a pin names another epoch. +3. `plain_ref_requires_complete_frame`: a plain ref targets a partial frame. +4. `ref_not_issued`: a pinned ref is outside the issuance set of a partial frame. -Each failure includes `details.ref`, `details.reason`, `details.currentGeneration`, the frame scope, -and `details.mintedGeneration` when the input is pinned. For example: +With the typed reasons, callers can distinguish "capture a complete snapshot" from "use the emitted pinned ref". +Each failure includes `details.ref`, `details.reason`, `details.currentGeneration`, and the frame scope. +When the input is pinned, the failure also includes `details.mintedGeneration`. +For example: ```text code: COMMAND_FAILED @@ -188,56 +222,63 @@ details.mintedGeneration: 42 # present only when the input was pinned hint: Capture a fresh interactive snapshot or use a stable selector. ``` -The daemon cannot truthfully report `mintedGeneration` for a plain ref. Error messages must not claim -that a found ref was missing or lacked bounds when the actual failure is lifetime expiration. +The daemon cannot truthfully report `mintedGeneration` for a plain ref. +When the actual failure is lifetime expiration, error messages must not claim that a found ref was missing or lacked bounds. ### Expiration occurs at the side-effect seam -Expiration occurs after syntax, capability, freshness, target resolution, replay identity, -occlusion, visibility, bounds, and other pre-action guards pass, but immediately before the first -operation that may change device-visible element identity. Persist the in-memory transition before -awaiting that operation. +Expiration occurs after these and all other pre-action guards pass: + +- Syntax. +- Capability. +- Freshness. +- Target resolution. +- Replay identity. +- Occlusion. +- Visibility. +- Bounds. + +Expiration occurs immediately before the first operation that may change device-visible element identity. +Before you await that operation, persist the in-memory transition. -Failure before this seam preserves the frame. Once the seam is crossed, success, timeout, -cancellation, connection loss, and ambiguous platform errors all leave it expired because the device -may already have acted. Crossing the seam also clears scoped-snapshot lineage. There is no success-only -rollback. +Failure before this seam preserves the frame. +After the seam is crossed, success, timeout, cancellation, connection loss, and ambiguous platform errors all leave the frame expired, because the device may already have acted. +A crossing of the seam also clears scoped-snapshot lineage. +There is no success-only rollback. This rule applies to every execution shape: -- runtime ref/selector/coordinate paths expire before `tap`, `fill`, `longPress`, text input, or - gesture execution; -- native web ref paths run their existing ref preflight first, then expire before `clickRef` or - `fillRef`; -- direct iOS selector paths consume no ref but expire any existing frame before the fused runner - resolve-and-act request; -- platform-generic, app lifecycle, keyboard, alert, settings, and provider actions expire at their - actual leaf side-effect seam, not at generic request admission; -- `open`/relaunch against an existing session expires that session's frame before the first - close/launch dispatch. A failure or timeout after dispatch leaves it expired; a successful `close` - deletes the session and therefore retains no frame to restore; -- a fallback or retry may proceed after an error only when the error proves no prior side effect; - ambiguous post-dispatch errors neither restore the frame nor authorize an automatic second action; -- Android blocking-dialog recovery is itself device-mutating. Its recovery tap/relaunch must cross - the same idempotent seam before acting, even when invoked from apparent readiness work. If recovery - mutates before a requested ref action dispatches, that ref action aborts with - `ref_frame_expired`; it cannot continue against the recovered UI. Selector and coordinate actions - may re-resolve and continue under their existing policies. - -The seam is deliberately inside leaf execution. Expiring at router entry would reject the ref needed -by the current command, invalidate on ordinary validation failures, and mishandle multiplexed and -orchestrating commands. - -Some platform operations fuse their final status/target check and mutation in one request, including -direct iOS selectors and current native alert or keyboard actions. Until a lower-level "will act" -callback exists, dispatching that fused request is the conservative seam. A later `not found`, -`already hidden`, timeout, or no-alert result is post-seam and does not restore the frame. A command -with a separate definitive read-only status request preserves the frame when that status proves no -mutation request was dispatched. +- Runtime ref/selector/coordinate paths expire before `tap`, `fill`, `longPress`, text input, or gesture execution. +- Native web ref paths run their existing ref preflight first, and then expire before `clickRef` or `fillRef`. +- Direct iOS selector paths consume no ref, but they expire any existing frame before the fused runner resolve-and-act request. +- Platform-generic, app lifecycle, keyboard, alert, settings, and provider actions expire at their actual leaf side-effect seam, not at generic request admission. +- `open`/relaunch against an existing session expires the frame of that session before the first close/launch dispatch. + A failure or timeout after dispatch leaves the frame expired. + A successful `close` deletes the session, and thus retains no frame to restore. +- After an error, a fallback or retry may proceed only when the error proves no prior side effect. + Ambiguous post-dispatch errors neither restore the frame nor authorize an automatic second action. +- Android blocking-dialog recovery is itself device-mutating. + Its recovery tap/relaunch must cross the same idempotent seam before it acts, even when apparent readiness work invokes the recovery. + If recovery mutates before a requested ref action dispatches, that ref action aborts with `ref_frame_expired`. + The ref action cannot continue against the recovered UI. + Selector and coordinate actions may re-resolve and continue under their existing policies. + +The seam is deliberately inside leaf execution. +Expiration at router entry would cause these problems: + +- It would reject the ref that the current command needs. +- It would invalidate on ordinary validation failures. +- It would mishandle multiplexed and orchestrating commands. + +Some platform operations fuse their final status/target check and their mutation in one request. +Examples are direct iOS selectors and current native alert or keyboard actions. +Until a lower-level "will act" callback exists, the dispatch of that fused request is the conservative seam. +A later `not found`, `already hidden`, timeout, or no-alert result is post-seam and does not restore the frame. +A command with a separate, definitive read-only status request preserves the frame when that status proves that no mutation request was dispatched. ### Command effects are daemon-owned and completely classified -The command descriptor's daemon facet declares a request policy: +The daemon facet of the command descriptor declares a request policy: ```ts type RefFrameEffect = 'preserve' | 'may-invalidate' | 'delegated'; @@ -245,233 +286,264 @@ type RefFrameEffect = 'preserve' | 'may-invalidate' | 'delegated'; type DaemonRefFrameEffect = RefFrameEffect | ((request: DaemonRequest) => RefFrameEffect); ``` -The daemon registry exposes a named resolver. Every daemon command is classified, but the -classification is an honesty/completeness guard rather than the transition site. `may-invalidate` -means that some successful path can cross a device side effect; the leaf implementation still calls -the ref-frame module only when that path is selected. Request-sensitive commands use a focused -resolver rather than pretending all subcommands behave alike. - -The registry is the exhaustive source of truth. The following table is non-exhaustive guidance for -classifying representative actions; it must not become a second prose registry: - -| Effect | Commands/actions | -| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `may-invalidate` | press, click, fill, longpress, type, focus, scroll, swipe, gesture, back, home, `tv-remote`, orientation, open/relaunch, trigger/push delivery, settings changes, install/reinstall, React Native overlay dismissal, and lifecycle operations that can replace the visible surface | -| Conditional resolver | keyboard status preserves while dismiss/return/input invalidate; alert get/wait preserve while accept/dismiss invalidate; find reads preserve while click/fill/focus/type delegate to their leaf mutation | -| `delegated` | batch, replay, and test/suite orchestrators; each nested leaf owns its transition | -| `preserve` | snapshots and other observation, assertion, screenshot, recording, trace, logs, events, network inspection, performance, inventory, capability, lease, and transport-management operations unless a selected subaction directly manipulates the visible surface | - -Clipboard reads and writes preserve the frame because pasteboard state alone does not change element -identity. A later paste/type action is independently invalidating. - -Generic routing is not an exception to the policy. `back`, `home`, `orientation`, `scroll`, `tv-remote`, -and `app-switcher` all reach the generic daemon leaf and must cross the same transition there when -they act. `app-switcher` is currently projected to the daemon by a direct writer but, unlike its -generic-routed siblings, omits an explicit daemon descriptor facet and relies on the registry's -generic fallback. Migration adds that facet and its effect classification; it does not invent a -specialized route. The completeness gate covers every command projected to the daemon, including -generic fallbacks, so a missing facet cannot hide an unclassified mutation. Mutations performed by -unrelated external tools remain outside this session guarantee. - -This policy is not derived from the runner-side classification. Apple runner command traits govern -retry eligibility, launch policy, and the recorded-failure conversion at a lower wire-command seam, -while the TypeScript `readOnly` trait is itself consumed as several daemon decisions: read-only -resend, session-invalidation skip, transport error classification, and readiness preflight. -`refFrameEffect` governs daemon session authorization and includes commands that never reach the -Apple runner. Narrow consistency tests may cover direct mappings, but blanket parity would couple -different concepts. - -Frame admission and transitions are serialized by the existing per-session request lock. The frame -is shared session state, not per-client or per-lease history. Generation pins make the rejected epoch -and reason precise; they do not reserve a frame or prevent another caller's mutation from expiring it. +The daemon registry exposes a named resolver. +Every daemon command is classified. +But the classification is an honesty/completeness guard, not the transition site. +`may-invalidate` means that some successful path can cross a device side effect. +The leaf implementation still calls the ref-frame module only when that path is selected. +Request-sensitive commands use a focused resolver, instead of a pretense that all subcommands behave alike. + +The registry is the exhaustive source of truth. +The table below is non-exhaustive guidance for the classification of representative actions. +The table must not become a second prose registry: + +| Effect | Commands/actions | +| --- | --- | +| `may-invalidate` | press, click, fill, longpress, type, focus, scroll, swipe, gesture, back, home, `tv-remote`, orientation, open/relaunch, trigger/push delivery, settings changes, install/reinstall, React Native overlay dismissal, and lifecycle operations that can replace the visible surface | +| Conditional resolver | keyboard status preserves while dismiss/return/input invalidate; alert get/wait preserve while accept/dismiss invalidate; find reads preserve while click/fill/focus/type delegate to their leaf mutation | +| `delegated` | batch, replay, and test/suite orchestrators; each nested leaf owns its transition | +| `preserve` | snapshots and other observation, assertion, screenshot, recording, trace, logs, events, network inspection, performance, inventory, capability, lease, and transport-management operations unless a selected subaction directly manipulates the visible surface | + +Clipboard reads and writes preserve the frame, because pasteboard state alone does not change element identity. +A later paste/type action is independently invalidating. + +Generic routing is not an exception to the policy. +`back`, `home`, `orientation`, `scroll`, `tv-remote`, and `app-switcher` all reach the generic daemon leaf. +When they act, they must cross the same transition there. +The completeness gate covers every command projected to the daemon, including generic fallbacks. +Thus, a missing facet cannot hide an unclassified mutation. +Mutations that unrelated external tools perform remain outside this session guarantee. + +This policy is not derived from the runner-side classification. +Apple runner command traits govern retry eligibility, launch policy, and the recorded-failure conversion at a lower wire-command seam. +The TypeScript `readOnly` trait is itself an input to several daemon decisions: + +- Read-only resend. +- Session-invalidation skip. +- Transport error classification. +- Readiness preflight. + +`refFrameEffect` governs daemon session authorization, and it includes commands that never reach the Apple runner. +Narrow consistency tests may cover direct mappings. +But blanket parity would couple different concepts. + +The existing per-session request lock serializes frame admission and transitions. +The frame is shared session state, not per-client or per-lease history. +Generation pins make the rejected epoch and reason precise. +They do not reserve a frame, and they do not prevent the mutation of another caller from expiring it. No second ref-specific mutex is introduced. ### Batch, replay, recording, and repeated actions -Batch and replay invoke their steps sequentially through ordinary daemon dispatch and are -`delegated`. They never expire a frame merely because the outer command began. Frame state naturally -carries between leaf steps: - -- `snapshot -> press @e1 -> press @e2` executes the first press and rejects the second; -- `snapshot -> press @e1 -> press @e1 -> press @e2` also stops on the second press, before `@e2`; -- `snapshot -> press @e1 -> snapshot -> press @e2` is admitted; -- selector-only and coordinate-only batches retain their request-round-trip advantage; and -- a stale-ref failure stops batch through its existing policy and preserves `step`, `command`, - `executed`, `total`, `partialResults`, diagnostic fields, and `details.reason`. - -Batch does not gain nested MCP pin traversal, intermediate-result interpolation, an unsafe -"assume stable refs" override, or a second static lifecycle interpreter. Normal leaf dispatch is the -single enforcement path. - -Recorded `.ad` scripts remain selector-first. The recorder continues converting successfully -resolved refs to selector chains and refuses to persist an unresolved session-bound bare ref. -Generation suffixes remain runtime evidence and are stripped from durable recordings because seeded -epochs are not portable across sessions. Legacy hand-written `.ad` scripts that mutate through -several bare refs from one snapshot become intentionally incompatible; they must capture between -mutations or use selectors. Replay target-binding verification completes before the side-effect seam; -identity divergence preserves the frame, while an action failure after dispatch leaves it expired. - -`press --count`, double-tap, hold, jitter, `swipe --count`, and platform sequence chunks are one -logical command. They resolve or plan once, expire once before the first repetition, and execute the -original physical point/plan for every repetition. They do not re-resolve the target or reactivate the -frame between repetitions. Failure in any later repetition or chunk leaves the frame expired. This -preserves the explicit repeated-physical-interaction contract while preventing later commands from -reusing the frame. +Batch and replay invoke their steps sequentially through ordinary daemon dispatch, and they are `delegated`. +They never expire a frame merely because the outer command began. +Frame state naturally carries between leaf steps: + +- `snapshot -> press @e1 -> press @e2` executes the first press and rejects the second. +- `snapshot -> press @e1 -> press @e1 -> press @e2` also stops on the second press, before `@e2`. +- `snapshot -> press @e1 -> snapshot -> press @e2` is admitted. +- Selector-only and coordinate-only batches retain their request-round-trip advantage. +- A stale-ref failure stops batch through its existing policy. + The failure preserves `step`, `command`, `executed`, `total`, `partialResults`, diagnostic fields, and `details.reason`. + +Batch does not gain any of these: + +- Nested MCP pin traversal. +- Intermediate-result interpolation. +- An unsafe "assume stable refs" override. +- A second static lifecycle interpreter. + +Normal leaf dispatch is the single enforcement path. + +Recorded `.ad` scripts remain selector-first. +The recorder continues to convert successfully resolved refs to selector chains. +It refuses to persist an unresolved session-bound bare ref. +Generation suffixes remain runtime evidence. +They are stripped from durable recordings, because seeded epochs are not portable across sessions. +Legacy hand-written `.ad` scripts that mutate through several bare refs from one snapshot become intentionally incompatible. +These scripts must capture between mutations or use selectors. +Replay target-binding verification completes before the side-effect seam. +Identity divergence preserves the frame, while an action failure after dispatch leaves it expired. + +Each of these is one logical command: + +- `press --count`. +- Double-tap. +- Hold. +- Jitter. +- `swipe --count`. +- Platform sequence chunks. + +Each one resolves or plans once, expires once before the first repetition, and executes the original physical point/plan for every repetition. +It does not re-resolve the target or reactivate the frame between repetitions. +Failure in any later repetition or chunk leaves the frame expired. +This preserves the explicit repeated-physical-interaction contract, and it prevents later commands from reuse of the frame. ### Partial issuance and MCP -`find`, settled diffs/tails, and replay divergence screens publish only the refs they actually return. +`find`, settled diffs/tails, and replay divergence screens publish only the refs that they actually return. They must not call a whole-frame "refs issued" marker. -Read-only `find` may partially publish its returned ref. A mutating `find` returns the acted ref only -as diagnostic pre-action identity: it must omit `refsGeneration`, must not publish or activate a -frame, and cannot make that ref valid after the nested action. - -The MCP executor continues storing per-ref pins by state-directory/session scope. Issuance handling -must distinguish the selected `find` action, not infer issuance from the command name alone. A -read-only `find` with `refsGeneration` merges its partial pin. A mutating `find` without -`refsGeneration` is explicitly non-issuing and leaves every remembered pin unchanged; it must not -enter a generic missing-generation branch that clears the session's pin scope. The executor must carry -enough normalized command/action context to make that distinction. Remembered pins are not cleared on -mutation: sending the remembered old epoch is how the daemon produces a precise stale rejection. MCP -currently neither traverses refs nested inside batch inputs nor learns refs from nested batch results; -this remains an explicit non-goal. - -The response-level `refsGeneration` field remains the token-economical authority. Snapshot nodes never -gain per-node generation bytes. Every reusable ref in a partial CLI text result must render in -ready-to-copy `@eN~s` form. JSON and Node.js responses retain plain ref bodies plus one -response-level generation; callers must pair them before a mutation. MCP may render a plain ref because -it stores and forwards the pin internally. Complete snapshot trees remain plain to protect the token -budget. A mutating `find` diagnostic ref is never rendered or stored as pinned because it carries no -`refsGeneration`. +Read-only `find` may partially publish its returned ref. +A mutating `find` returns the acted ref only as diagnostic pre-action identity: + +- It must omit `refsGeneration`. +- It must not publish or activate a frame. +- It cannot make that ref valid after the nested action. + +The MCP executor continues to store per-ref pins by state-directory/session scope. +Issuance handling must distinguish the selected `find` action. +It must not infer issuance from the command name alone. +A read-only `find` with `refsGeneration` merges its partial pin. +A mutating `find` without `refsGeneration` is explicitly non-issuing, and it leaves every remembered pin unchanged. +It must not enter a generic missing-generation branch that clears the pin scope of the session. +The executor must carry sufficient normalized command/action context to make that distinction. +Remembered pins are not cleared on mutation. +The daemon produces a precise stale rejection from the remembered old epoch that the executor sends. +MCP currently does not traverse refs nested inside batch inputs, and it does not learn refs from nested batch results. +This remains an explicit non-goal. + +The response-level `refsGeneration` field remains the token-economical authority. +Snapshot nodes never gain per-node generation bytes. +Every reusable ref in a partial CLI text result must render in the ready-to-copy `@eN~s` form. +JSON and Node.js responses retain plain ref bodies plus one response-level generation. +Before a mutation, callers must pair them. +MCP may render a plain ref, because it stores and forwards the pin internally. +Complete snapshot trees remain plain to protect the token budget. +A mutating `find` diagnostic ref is never rendered or stored as pinned, because it carries no `refsGeneration`. ## Performance and compatibility -Expiration is O(1): one monotonic epoch transition and bounded in-memory state. It adds no capture, -platform request, daemon round trip, or per-node output. The model retains at most one authorized frame -and one latest operational observation, sharing immutable capture objects rather than deep copies; it -never caches an unbounded history of full trees. Existing capture size limits remain the bound. Memory -pressure must not silently expire or swap a frame because that would turn deterministic admission into -environment-dependent behavior. - -The real cost is workflow-level and intentional. A ref-oriented sequence that performs several -mutations may need an explicit observation between them. Callers can avoid unnecessary captures by -using selectors, direct selector fast paths, a settled partial result whose emitted ref is pinned, or -`--count` for a repeated physical action at one resolved target. The daemon never auto-captures merely -to renew refs. - -Older clients remain wire-parse-compatible: command grammar, `refsGeneration`, and `~s` remain -unchanged, and structured error details are additive, so ADR 0006 requires no protocol bump. They are -not fully behaviorally compatible. Unsafe plain-ref mutation chains newly fail. A fused command can -also expire the frame when its eventual result is a no-op, such as `keyboard dismiss` reporting that -the keyboard was already hidden or an alert action reporting no alert. Read-only refs that lack -retained frame evidence newly fail instead of warning and resolving by positional coincidence against -a newer observation. These changes require CLI/help, changelog, `.ad` migration, and batch guidance. - -It composes with existing ADRs as follows: - -- ADR 0003/0008: daemon-owned effect policy is composed in the descriptor daemon facet and projected - only through the daemon registry; -- ADR 0004: capture plans and sparse-quality rules are unchanged, and no capture is added; -- ADR 0005: runner liveness/preflight state remains independent from session ref authorization; -- ADR 0006/0010: no RPC bump; typed `details.reason` and actionable hints follow the error contract; -- ADR 0007: the frame is session-scoped under the existing session/request lock, not client- or - lease-local; -- ADR 0009: one target model applies to Apple, Android, Linux, and web while public platform output - remains projected; -- ADR 0011: lifetime is a sibling session-sequence invariant, not another path-by-element-guarantee - cell; every mutating path still crosses the shared seam; -- ADR 0012: durable target identity remains selector/evidence based, and divergence refs become - honest partial issuance; and +Expiration is O(1): one monotonic epoch transition and bounded in-memory state. +It adds no capture, platform request, daemon round trip, or per-node output. +The model retains at most one authorized frame and one latest operational observation. +These share immutable capture objects, not deep copies. +The model never caches an unbounded history of full trees. +Existing capture size limits remain the bound. +Memory pressure must not silently expire or swap a frame, because that would make deterministic admission depend on the environment. + +The real cost is at the workflow level, and it is intentional. +A ref-oriented sequence that performs several mutations may need an explicit observation between them. +Callers can avoid unnecessary captures with: + +- Selectors. +- Direct selector fast paths. +- A settled partial result whose emitted ref is pinned. +- `--count` for a repeated physical action at one resolved target. + +The daemon never captures automatically for the sole purpose of a ref renewal. + +Older clients remain wire-parse-compatible: + +- Command grammar, `refsGeneration`, and `~s` remain unchanged. +- Structured error details are additive. + +Thus, ADR 0006 requires no protocol bump. +But older clients are not fully compatible in behavior: + +- Unsafe plain-ref mutation chains newly fail. +- A fused command can also expire the frame when its eventual result is a no-op. + Examples are `keyboard dismiss` that reports an already hidden keyboard, and an alert action that reports no alert. +- Read-only refs that lack retained frame evidence newly fail, instead of a warning and a resolution by positional coincidence against a newer observation. + +These changes require CLI/help, changelog, `.ad` migration, and batch guidance. + +This decision composes with existing ADRs as follows: + +- ADR 0003/0008: daemon-owned effect policy is composed in the descriptor daemon facet and projected only through the daemon registry. +- ADR 0004: capture plans and sparse-quality rules are unchanged, and no capture is added. +- ADR 0005: runner liveness/preflight state remains independent from session ref authorization. +- ADR 0006/0010: there is no RPC bump; typed `details.reason` and actionable hints follow the error contract. +- ADR 0007: the frame is session-scoped under the existing session/request lock, not client-local or lease-local. +- ADR 0009: one target model applies to Apple, Android, Linux, and web, while public platform output remains projected. +- ADR 0011: lifetime is a sibling session-sequence invariant, not another path-by-element-guarantee cell. + Every mutating path still crosses the shared seam. +- ADR 0012: durable target identity remains selector/evidence based, and divergence refs become honest partial issuance. - ADR 0013: coordinate gestures expire frames but remain outside the element guarantee matrix. ## Consequences -- A stale ref cannot silently mutate a different element after navigation; the caller receives a - typed failure and must choose a fresh observation or a selector. -- Ref-oriented workflows that perform multiple logical mutations must capture again between them, - consume an honestly issued settled ref, use selectors, or use one repeated-action command such as - `--count`. Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must - change. -- Conservative fused requests may consume ref authority even when the platform later reports that no - action was needed. This favors avoiding a wrong-screen action over preserving refs after an - ambiguous dispatch. -- A read without retained evidence fails rather than resolving the same positional ref body in a - newer tree. Reads with retained evidence keep the structured warning behavior. -- Consecutive repeated scoped snapshots retain their admitted semantic scope, while any mutation, - unrelated publication, or different scope breaks that lineage. -- The implementation pays bounded session-memory bookkeeping but adds no automatic capture, platform - call, round trip, or per-node response bytes. +- A stale ref cannot silently mutate a different element after navigation. + The caller receives a typed failure and must choose a fresh observation or a selector. +- Ref-oriented workflows that perform multiple logical mutations must capture again between the mutations, consume an honestly issued settled ref, use selectors, or use one repeated-action command such as `--count`. + Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must change. +- Conservative fused requests may consume ref authority even when the platform later reports that no action was necessary. + This favors the avoidance of a wrong-screen action over the preservation of refs after an ambiguous dispatch. +- A read without retained evidence fails, instead of resolving the same positional ref body in a newer tree. + Reads with retained evidence keep the structured warning behavior. +- Consecutive repeated scoped snapshots retain their admitted semantic scope. + Any mutation, unrelated publication, or different scope breaks that lineage. +- The implementation pays for bounded bookkeeping in session memory. + But it adds no automatic capture, platform call, round trip, or per-node response bytes. ## Required evidence -The implementation is not complete until contract and provider tests prove: - -1. a bare and pinned second mutation reject after an unobserved first mutation; -2. definitive pre-seam validation, target, and replay-identity failures preserve the frame; -3. timeout, cancellation, connection loss, and other post-seam failures leave it expired; -4. runtime ref/selector/coordinate, native ref, direct selector, provider-backed interaction, and - provider-backed lifecycle/generic paths cross the same seam; -5. Android freshness cannot retarget an admitted ref through positional coincidence; -6. read-only operational captures do not replace the authorized frame; -7. partial publication admits only emitted pinned refs, reports each typed admission failure, cannot - bless unrelated plain refs, and CLI text renders every reusable partial ref with its pin while JSON - and Node.js retain the response-level representation; -8. complete publication admits its plain namespace while sparse/unusable captures do not; -9. mutating `find` omits `refsGeneration` and never pins or activates its pre-action ref as current - post-action state; an MCP sequence that first remembers a snapshot pin, then performs mutating - `find`, still forwards the remembered old pin on a later ref input so the daemon rejects it as stale; -10. batch and replay delegate per step, retain their existing failure/partial-result contracts, and - allow `snapshot -> @ref` with an intermediate digest without exposing pins or dynamic bindings; -11. recorded scripts contain selectors rather than portable generation claims; -12. press/swipe series and Apple chunks resolve and expire exactly once; -13. Android dialog recovery expires before its first recovery side effect and aborts an outstanding - ref action after recovery mutates; -14. consecutive repeated `snapshot -s @ref` calls resolve the same admitted semantic scope after - scoped output reindexes refs; a mutation, unrelated publication, or different scope breaks the - lineage; an arbitrary stale ref cannot use it; and empty/unusable repetition preserves the last - useful frame and still-consecutive lineage; -15. fused keyboard/alert requests and split status/action paths follow their documented conservative - seams, including already-hidden and no-alert outcomes; -16. failed or timed-out existing-session open/relaunch leaves the old frame expired after dispatch; -17. a read-only ref with retained stale evidence returns the structured warning, while missing frame - evidence fails without falling through to a newer observation by positional coincidence; -18. a large-tree fixture proves frame/observation separation introduces no deep copy and records the - bounded peak retained nodes/bytes; and -19. enforcement and error shapes are consistent across supported platforms. - -Performance regression tests prove that admission, expiration, and frame/observation separation add -no implicit capture or provider/platform call. Device benchmarks are required only if those tests or -live evidence reveal a regression. No path may claim success until a test proves the relevant -direct/native/runtime/provider branch actually executed. - -Before enforcement is enabled for a platform, fresh-build live evidence must exercise every supported -production seam affected there: Apple runtime-ref and direct/native paths where available, Android -helper freshness and recovery, generic/lifecycle mutation, and at least one real provider-backed -interaction plus lifecycle operation. Each run must prove a fresh ref succeeds, a second mutation with -the stale ref is rejected before dispatch, and a fresh observation restores usability. A seam not -exercised remains disabled or is recorded as an explicit release blocker. Fixture-backed tests and -registry claims are necessary but do not substitute for this live evidence. +The implementation is not complete until contract and provider tests prove all of these: + +1. After an unobserved first mutation, a bare and a pinned second mutation reject. +2. Definitive pre-seam validation, target, and replay-identity failures preserve the frame. +3. Timeout, cancellation, connection loss, and other post-seam failures leave the frame expired. +4. These paths cross the same seam: runtime ref/selector/coordinate, native ref, direct selector, provider-backed interaction, and provider-backed lifecycle/generic paths. +5. Android freshness cannot retarget an admitted ref through positional coincidence. +6. Read-only operational captures do not replace the authorized frame. +7. Partial publication admits only emitted pinned refs, reports each typed admission failure, and cannot bless unrelated plain refs. + CLI text renders every reusable partial ref with its pin, while JSON and Node.js retain the response-level representation. +8. Complete publication admits its plain namespace, while sparse/unusable captures do not. +9. Mutating `find` omits `refsGeneration` and never pins or activates its pre-action ref as current post-action state. + An MCP sequence that first remembers a snapshot pin and then performs mutating `find` still forwards the remembered old pin on a later ref input. + Thus, the daemon rejects that pin as stale. +10. Batch and replay delegate per step and retain their existing failure/partial-result contracts. + They allow `snapshot -> @ref` with an intermediate digest, without exposed pins or dynamic bindings. +11. Recorded scripts contain selectors, not portable generation claims. +12. Press/swipe series and Apple chunks resolve and expire exactly once. +13. Android dialog recovery expires before its first recovery side effect. + After recovery mutates, the recovery aborts an outstanding ref action. +14. Consecutive repeated `snapshot -s @ref` calls resolve the same admitted semantic scope after scoped output reindexes refs. + A mutation, an unrelated publication, or a different scope breaks the lineage. + An arbitrary stale ref cannot use the lineage. + Empty/unusable repetition preserves the last useful frame and the still-consecutive lineage. +15. Fused keyboard/alert requests and split status/action paths follow their documented conservative seams, including already-hidden and no-alert outcomes. +16. A failed or timed-out existing-session open/relaunch leaves the old frame expired after dispatch. +17. A read-only ref with retained stale evidence returns the structured warning. + Missing frame evidence fails, without a fall-through to a newer observation by positional coincidence. +18. A large-tree fixture proves that frame/observation separation introduces no deep copy. + The fixture records the bounded peak of retained nodes/bytes. +19. Enforcement and error shapes are consistent across supported platforms. + +Performance regression tests prove that admission, expiration, and frame/observation separation add no implicit capture or provider/platform call. +Device benchmarks are required only if those tests or live evidence reveal a regression. +No path may claim success until a test proves that the relevant direct/native/runtime/provider branch actually executed. + +Before enforcement is enabled for a platform, fresh-build live evidence must exercise every supported production seam affected on that platform: + +- Apple runtime-ref and direct/native paths, where available. +- Android helper freshness and recovery. +- Generic/lifecycle mutation. +- At least one real provider-backed interaction plus lifecycle operation. + +Each run must prove all of these: + +1. A fresh ref succeeds. +2. A second mutation with the stale ref is rejected before dispatch. +3. A fresh observation restores usability. + +A seam that is not exercised remains disabled or is recorded as an explicit release blocker. +Fixture-backed tests and registry claims are necessary, but they do not substitute for this live evidence. ## Alternatives considered -- **Stable refs across trees:** rejected because positional accessibility snapshots have no durable - cross-tree identity, and semantic IDs are neither universal nor unique. -- **Expire only after successful mutation:** rejected because a timeout or transport failure may occur - after the device acted. -- **Keep one snapshot for observation and authorization:** rejected because read-only internal captures - can reindex or invalidate caller refs. -- **Automatically capture after every mutation:** rejected because it adds the dominant platform cost, - can itself fail or be sparse, and hides rather than defines lifetime. -- **Retain every generation's full tree:** rejected as unbounded session memory and unnecessary for a - single-current-frame contract. -- **Add a per-ref historical ledger immediately:** rejected until evidence requires concurrent - generation support; one bounded current frame plus issuance scope is sufficient. -- **Derive the policy from the runner's command traits:** rejected because those traits classify a - wire command for what the runner may still do about its app, while daemon ref authorization - classifies a command for what it did to the frame. -- **Add batch interpolation or an unsafe ref-stability override:** rejected as a new orchestration - interface that bypasses the same safety rule. -- **Force lifetime into ADR 0011's element path matrix:** rejected because ref lifetime spans commands, - time, non-element invalidators, and orchestrators; the matrix remains the enforcement record for - per-path element guarantees. +- **Stable refs across trees.** Rejected because positional accessibility snapshots have no durable cross-tree identity. + Semantic IDs are neither universal nor unique. +- **Expire only after successful mutation.** Rejected because a timeout or transport failure may occur after the device acted. +- **Keep one snapshot for observation and authorization.** Rejected because read-only internal captures can reindex or invalidate caller refs. +- **Automatically capture after every mutation.** Rejected because it adds the dominant platform cost, and it can itself fail or be sparse. + It also hides lifetime, instead of defining it. +- **Retain every generation's full tree.** Rejected because it uses unbounded session memory. + It is also unnecessary for a single-current-frame contract. +- **Add a per-ref historical ledger immediately.** Rejected until evidence requires concurrent generation support. + One bounded current frame plus issuance scope is sufficient. +- **Derive the policy from the runner's command traits.** Rejected because those traits classify a wire command for what the runner may still do about its app. + Daemon ref authorization classifies a command for what the command did to the frame. +- **Add batch interpolation or an unsafe ref-stability override.** Rejected as a new orchestration interface that bypasses the same safety rule. +- **Force lifetime into ADR 0011's element path matrix.** Rejected because ref lifetime spans commands, time, non-element invalidators, and orchestrators. + The matrix remains the enforcement record for per-path element guarantees. diff --git a/docs/adr/0015-direct-maestro-engine.md b/docs/adr/0015-direct-maestro-engine.md index ac124de178..68a142c2b5 100644 --- a/docs/adr/0015-direct-maestro-engine.md +++ b/docs/adr/0015-direct-maestro-engine.md @@ -4,220 +4,258 @@ Accepted +## Rules at a glance + +- Parse supported Maestro YAML into a source-preserving typed program. Execute it directly through a narrow compatibility runtime port. +- Do not lower the program through `SessionAction`, private command names, positional JSON, or a recursive replay dispatcher. +- One explicit execution context owns variables and the current observation generation. +- Every mutating attempt invalidates retained evidence before dispatch, including an attempt whose dispatch reports failure. +- Keep geometry action-local. Never carry it across command boundaries. +- Preserve upstream first-match and explicit-index semantics. Do not adopt the unique-match requirement of the public command surface. +- Maestro `optional` does not suppress an `AMBIGUOUS_MATCH` from a nested public command. +- Before another mutating command, settle the pending stabilization boundary. Commands with specialized completion semantics settle inline instead. +- An authored observation after a mutation skips the pending barrier and polls its own condition immediately. +- Preserve the performance contract. Encode documented intentional deviations as declared expected-divergence entries. +- Keep `${...}` interpolation variable-lookup-only. Gate `evalScript` with `MaestroEngineOptions.trustedScripts`. +- Do not ship two production engines or a runtime fallback between them. + ## Context -At decision time, Maestro YAML compiled into generic `SessionAction` strings. Commands whose semantics did not -match native replay become private `__maestro*` names with positional payloads, then replay dispatch -routes those strings through a second Maestro switch before recursively invoking ordinary daemon -commands. Compatibility state is split between the replay variable scope and three `WeakMap` caches. +At decision time, Maestro YAML compiled into generic `SessionAction` strings. +Commands whose semantics did not match native replay became private `__maestro*` names with positional payloads. +Replay dispatch then routed those strings through a second Maestro switch before it recursively invoked ordinary daemon commands. +Compatibility state was split between the replay variable scope and three `WeakMap` caches. -That path grew a second selector resolver, several polling loops, action-after-assertion recovery, and -gesture-coordinate adapters. The indirection makes successful responses hard to prove: a fast native -wait may establish text existence while the compatibility assertion requires a visible Maestro target. -It also hides authored coordinate space until runtime positional decoding. +That path grew a second selector resolver, several polling loops, action-after-assertion recovery, and gesture-coordinate adapters. +The indirection makes successful responses hard to prove. +For example, a fast native wait may establish text existence while the compatibility assertion requires a visible Maestro target. +The indirection also hides authored coordinate space until runtime positional decoding. -The inbound implementation was about 5,000 production lines and 5,000 focused test lines. -Android compatibility is materially faster than native Maestro in the pager and navigation suites; -that advantage is a product constraint, not expendable migration headroom. +The inbound implementation was approximately 5,000 production lines and 5,000 focused test lines. +Android compatibility is materially faster than native Maestro in the pager and navigation suites. +That advantage is a product constraint, not expendable migration headroom. -ADR 0013 separately owns public gesture normalization, contact topology, trajectory planning, and -native injection. Maestro's supported gesture surface is single-pointer swipe only. Maestro swipes -normalize to ADR 0013's canonical `pan` input (origin, delta, durationMs) and carry the -`endpoint-hold` execution profile through a daemon-internal compatibility seam -(`internal.gestureExecutionProfile`), preserving iOS fast-swipe-then-hold behavior without exposing -the profile on the public command surface. +ADR 0013 separately owns public gesture normalization, contact topology, trajectory planning, and native injection. +Maestro's supported gesture surface is single-pointer swipe only. +Maestro swipes normalize to ADR 0013's canonical `pan` input (origin, delta, durationMs). +They carry the `endpoint-hold` execution profile through a daemon-internal compatibility seam (`internal.gestureExecutionProfile`). +This preserves iOS fast-swipe-then-hold behavior, and the public command surface does not expose the profile. ## Decision -Parse supported Maestro YAML into a source-preserving typed program and execute it directly through a -narrow compatibility runtime port. Do not lower the program through `SessionAction`, private command -names, positional JSON, or a recursive replay dispatcher. The daemon adapter may invoke ordinary public -commands as typed operations; those calls reuse the shared command semantics and never re-enter Maestro -parsing or compatibility dispatch. +Parse supported Maestro YAML into a source-preserving typed program. +Execute the program directly through a narrow compatibility runtime port. +Do not lower the program through `SessionAction`, private command names, positional JSON, or a recursive replay dispatcher. +The daemon adapter may invoke ordinary public commands as typed operations. +Those calls reuse the shared command semantics and never re-enter Maestro parsing or compatibility dispatch. The engine has five responsibilities: 1. The parser validates the supported grammar and preserves source path and line on every command. -2. The interpreter owns hooks, includes, environment scopes, conditions, repeat/retry, and ordered - command execution. Runtime repeat/retry blocks remain compact plan nodes; the plan does not expand - authored iteration counts. `repeat.times` preserves the authored count, while retry clamps - `maxRetries` to upstream's three-retry preset (four total attempts). -3. The runtime port exposes typed app, input, observation, target, and single-pointer gesture - operations backed by the existing agent-device runtime and platform adapters. -4. One explicit execution context owns variables and the current observation generation. A mutation - invalidates that generation; reads may reuse semantic evidence only within the same generation. +2. The interpreter owns hooks, includes, environment scopes, conditions, repeat/retry, and ordered command execution. + Runtime repeat/retry blocks remain compact plan nodes. + The plan does not expand authored iteration counts. + `repeat.times` preserves the authored count. + Retry clamps `maxRetries` to upstream's three-retry preset (four total attempts). +3. The runtime port exposes typed app, input, observation, target, and single-pointer gesture operations. + The existing agent-device runtime and platform adapters back these operations. +4. One explicit execution context owns variables and the current observation generation. + A mutation invalidates that generation. + Reads may reuse semantic evidence only within the same generation. Geometry is action-local and is never carried across command boundaries. -5. An observer adapts source-aware progress, traces, artifacts, and failures to the existing replay and - test result contracts. Observer telemetry is redacted and best-effort; trace persistence cannot - change command success or failure. - -Maestro `stopApp` terminates its target through a daemon-internal app-only close and preserves the -owning replay session. Public `close` remains the session finalizer that releases recordings, runner -state, leases, and stored runtime hints. - -The engine does not implement platform input. Absolute and percentage swipes preserve authored -endpoints without a hierarchy capture. Directional horizontal swipes reuse ADR 0013's shared in-page -preset geometry so an iOS right swipe does not become an interactive-back gesture; vertical presets -retain Maestro's platform geometry. All viewport-relative swipes resolve the cheapest fresh interaction -viewport available so ADR 0013 can validate every planned sample. Maestro swipes pair that resolved -viewport with `internal.gestureExecutionProfile: 'endpoint-hold'` on the nested public `gesture` request -so ADR 0013 produces the iOS fast-swipe-then-hold profile without a public `executionProfile` field. -When normalization already resolves a viewport, the adapter pairs it with the nested public gesture -request as daemon-internal metadata. ADR 0013 planning consumes that exact frame instead of probing -the platform a second time. - -Target-relative swipes reuse the target-resolution observation. The resulting typed single-pointer -motion enters ADR 0013 as the canonical `pan` input, with the `endpoint-hold` execution profile carried -as daemon-internal metadata. Maestro code cannot construct or execute two-pointer pan, pinch, rotate, -transform, or physical pointer trajectories. - -Simple successful target queries return their match, visibility decision, candidate count, and -observation generation in one response. The daemon may retain the provider snapshot behind that -evidence, but geometry stays action-local. Raw hierarchies, screenshots, and complete candidate lists -are failure/debug artifacts, not happy-path requirements. - -The daemon adapter may retain the provider snapshot behind a successful observation without exposing it -through the engine contract. A following target resolution may use that snapshot only as semantic -evidence. Maestro selector semantics use the shared structurally normalized provider snapshot, before -iOS canonical interactive presentation. They deliberately do not request the public `snapshot --raw` -shape, whose output contract bypasses shared pruning and overlay annotations. On iOS, the adapter derives the provider's canonical -interactive presentation from that same snapshot without another capture. Raw matching still selects -the node and authored index; when that same source node has canonical interactive bounds, tap-like -actions use those bounds. A unique exact, canonically hittable match may be dispatched with its -resolved point so XCTest binds the live selector identity and coordinate delivery atomically; a -structured live-selector miss, ambiguity, point mismatch, or off-screen result -falls back to fresh Maestro resolution. All other targets capture fresh geometry before coordinate -dispatch. Maestro visibility does not apply agent-device's React Native overlay blocking policy; -debug-overlay handling remains an explicit public-command workflow. Visibility can -be true while a scroll view or tab strip is still moving, so an observation frame is never authoritative -for a later interaction, even within the same mutation generation. Every mutating attempt invalidates -retained evidence before dispatch, including an attempt whose dispatch reports failure, because the -adapter cannot prove the app stayed unchanged. - -Maestro target resolution preserves upstream first-match and explicit-index semantics. Equivalent iOS -accessibility wrapper chains are normalized to the semantic control or deepest matching leaf before -applying an authored index, matching XCTest element-query behavior without changing distinct nested -matches. It does not adopt the public agent-device command surface's unique-match requirement. Conversely, an -`AMBIGUOUS_MATCH` produced by a nested public command is an agent-device authoring failure and is not -suppressed by Maestro `optional`; atomic iOS dispatch handles that result by performing fresh Maestro -resolution, as described above. Cancellation and infrastructure failures are likewise non-optional. - -Mutations that can leave an in-flight transition record a pending stabilization boundary. Before another -mutating command, the runtime samples at the compatibility polling cadence until the hierarchy is stable -and retains that final observation for the next read. Commands with specialized completion semantics settle -inline instead. Hierarchy signatures project the available semantic attributes, tree topology, and -fixed-order integer edges while excluding provider-only metadata, object-key order, and subpixel noise. +5. An observer adapts source-aware progress, traces, artifacts, and failures to the existing replay and test result contracts. + Observer telemetry is redacted and best-effort. + Trace persistence cannot change command success or failure. + +Maestro `stopApp` terminates its target through a daemon-internal app-only close and preserves the owning replay session. +Public `close` remains the session finalizer that releases recordings, runner state, leases, and stored runtime hints. + +The engine does not implement platform input. +Absolute and percentage swipes preserve authored endpoints without a hierarchy capture. +Directional horizontal swipes reuse ADR 0013's shared in-page preset geometry. +Thus, an iOS right swipe does not become an interactive-back gesture. +Vertical presets retain Maestro's platform geometry. +All viewport-relative swipes resolve the least expensive fresh interaction viewport available, so that ADR 0013 can validate every planned sample. +Maestro swipes pair that resolved viewport with `internal.gestureExecutionProfile: 'endpoint-hold'` on the nested public `gesture` request. +Thus, ADR 0013 produces the iOS fast-swipe-then-hold profile without a public `executionProfile` field. +If normalization already resolves a viewport, the adapter pairs it with the nested public gesture request as daemon-internal metadata. +ADR 0013 planning consumes that exact frame and does not probe the platform a second time. + +Target-relative swipes reuse the target-resolution observation. +The resulting typed single-pointer motion enters ADR 0013 as the canonical `pan` input. +The `endpoint-hold` execution profile is carried as daemon-internal metadata. +Maestro code cannot construct or execute two-pointer pan, pinch, rotate, transform, or physical pointer trajectories. + +Simple successful target queries return their match, visibility decision, candidate count, and observation generation in one response. +The daemon may retain the provider snapshot behind that evidence, but geometry stays action-local. +Raw hierarchies, screenshots, and complete candidate lists are failure/debug artifacts, not happy-path requirements. + +The daemon adapter may retain the provider snapshot behind a successful observation without exposing it through the engine contract. +A following target resolution may use that snapshot only as semantic evidence. +Maestro selector semantics use the shared structurally normalized provider snapshot, before iOS canonical interactive presentation. +They deliberately do not request the public `snapshot --raw` shape, whose output contract bypasses shared pruning and overlay annotations. +On iOS, the adapter derives the provider's canonical interactive presentation from that same snapshot without another capture. +Raw matching still selects the node and authored index. +If that same source node has canonical interactive bounds, tap-like actions use those bounds. +A unique exact, canonically hittable match may be dispatched with its resolved point. +XCTest then binds the live selector identity and coordinate delivery atomically. +A structured live-selector miss, ambiguity, point mismatch, or off-screen result falls back to fresh Maestro resolution. +All other targets capture fresh geometry before coordinate dispatch. +Maestro visibility does not apply agent-device's React Native overlay blocking policy. +Debug-overlay handling remains an explicit public-command workflow. +Visibility can be true while a scroll view or tab strip is still in motion. +Thus, an observation frame is never authoritative for a later interaction, even within the same mutation generation. +Every mutating attempt invalidates retained evidence before dispatch. +This includes an attempt whose dispatch reports failure, because the adapter cannot prove that the app stayed unchanged. + +Maestro target resolution preserves upstream first-match and explicit-index semantics. +Before an authored index applies, equivalent iOS accessibility wrapper chains are normalized to the semantic control or the deepest matching leaf. +This matches XCTest element-query behavior and does not change distinct nested matches. +Maestro target resolution does not adopt the unique-match requirement of the public agent-device command surface. +Conversely, an `AMBIGUOUS_MATCH` from a nested public command is an agent-device authoring failure. +Maestro `optional` does not suppress it. +Atomic iOS dispatch handles that result with fresh Maestro resolution, as described above. +Cancellation and infrastructure failures are likewise non-optional. + +Mutations that can leave an in-flight transition record a pending stabilization boundary. +Before another mutating command, the runtime samples at the compatibility polling cadence until the hierarchy is stable. +The runtime retains that final observation for the next read. +Commands with specialized completion semantics settle inline instead. +Hierarchy signatures project the available semantic attributes, tree topology, and fixed-order integer edges. +They exclude provider-only metadata, object-key order, and subpixel noise. An authored observation after a mutation skips the pending barrier and polls its own condition immediately. -Its successful snapshot becomes the baseline for the boundary but does not discharge mutation ordering. If -another mutation follows, the runtime waits one polling interval and compares a fresh hierarchy with that -baseline, continuing until stable when the UI is still changing. If no later mutation follows, it performs no -extra capture. This intentionally differs from upstream Maestro, which settles before evaluating an assertion, -and retains the previous agent-device compatibility engine's faster read behavior without allowing a later -tap, gesture, scroll, or input to overtake an in-flight transition. - -`retryTapIfNoChange` defaults to false. When explicitly enabled, the runtime compares the stable hierarchy -with the target-resolution hierarchy. On iOS, an unchanged hierarchy triggers the same screenshot comparison -used by Maestro; the runtime retries once only when both surfaces remain unchanged. Screenshot evidence is -best-effort because a failed capture cannot prove that repeating a mutation is safe. The stable hierarchy -primes the next command, so the retry policy does not add another hierarchy read there. - -`waitForAnimationToEnd` uses its own screenshot-stability operation, matching upstream's two immediate -captures per attempt and 0.005% normalized absolute RGB-difference threshold. These captures explicitly -bypass ordinary screenshot stabilization so the command observes the application rather than -recursively waiting on another settling policy. After a successful authored visual wait, screenshot -stability discharges the same-generation pending mutation boundary instead of preceding the visual -barrier with a hierarchy capture. Reaching the visual deadline without observing a stable pair leaves -that boundary pending for the next mutating command's hierarchy settle. On iOS, -comparison captures use the persistent runner's screenshot surface so both frames come from one warmed -transport and avoid simulator screenshot setup between polls. - -Upstream Maestro is a version-pinned development reference, not a production dependency. Conformance is -enforced by the three-layer oracle in `scripts/maestro-conformance` (issue #1274), which replaced the -original hand-typed parser fixture. Layer 1 drives the pinned `maestro-orchestra` YAML parser -(`dev.mobile:maestro-orchestra:2.5.1`, SHA-verified at regeneration) over a corpus of vendored upstream -flows plus targeted bug-class flows, capturing each parse; the deterministic verifier replays that -capture against our engine and classifies every flow as identical / both-reject / we-reject / mismatch, -requiring each non-identical outcome to be a declared, on-the-record divergence. Layer 2 reads upstream -constants (retry cap, animation-wait threshold/timeout, erase cap) and parser-observed defaults (swipe -duration) straight from the pinned bytecode and cross-checks each against the live `MAESTRO_COMPATIBILITY_PRESETS` -constant. Both generated layers are checked in and verified in CI via `node --test` with no Java. Because -per-PR CI cannot re-derive them, each fixture carries a `contentHash` seal that the verifier recomputes, -so a hand edit to a captured command or constant fails the build; the scheduled `conformance-regenerate` -job then re-runs the harness against the pinned jars and fails on any byte difference, which is what makes -"generated from upstream" enforced rather than documented. Layer 3 runs a small set of app-observable -differential scenarios (settle ordering, tap retries, optional warned-vs-failed) through both engines -against the real fixture app, pinning the Maestro CLI to the same version as layers 1-2; it is scheduled, -never per-PR. Layer 3 carries the same declared-divergence contract as layer 1: a scenario that currently -fails declares a `knownDivergence` with a required tracking issue, the schedule stays green on that known -gap, and only undeclared divergences fail. A declaration that stops reproducing also fails, so the fix PR -must remove it and the differential becomes the acceptance test for its own findings. This exists because -the instrument must not be blocked on repairing what it just measured. The four #1217 regressions (percent -rounding/rejection, target-swipe direction default, retry-cap semantics, settle-loop ordering) each have a -dedicated fixture. Percent truncation's runtime half is pinned by a pure unit test rather than a device -scenario, because a one-pixel delta is not app-observable. Bug class 4's detector is an engine-side timing -invariant, unit-tested against synthetic traces, that executes on the device path — currently blocked by a -declared `scrollUntilVisible` divergence (#1299) that the differential itself found on its first working run. - -Documented intentional deviations are encoded as expected-divergence entries rather than silent -mismatches: the hierarchy-vs-screenshot animation wait (constants match, mechanism differs), the -horizontal-swipe in-page presets, condition-poll-as-stabilization, ambiguity strictness under `optional`, -and the omitted iOS 3s pre-tap static gate (`IOSDriver.SCREEN_SETTLE_TIMEOUT_MS`, a layer-2 reference-only -vector). +Its successful snapshot becomes the baseline for the boundary but does not discharge mutation ordering. +If another mutation follows, the runtime waits one polling interval and compares a fresh hierarchy with that baseline. +If the UI is still changing, the runtime continues until the hierarchy is stable. +If no later mutation follows, the runtime performs no extra capture. +This intentionally differs from upstream Maestro, which settles before it evaluates an assertion. +It retains the faster read behavior of the previous agent-device compatibility engine. +It does not allow a later tap, gesture, scroll, or input to overtake an in-flight transition. + +`retryTapIfNoChange` defaults to false. +If it is explicitly enabled, the runtime compares the stable hierarchy with the target-resolution hierarchy. +On iOS, an unchanged hierarchy triggers the same screenshot comparison that Maestro uses. +The runtime retries once, and only if both surfaces remain unchanged. +Screenshot evidence is best-effort because a failed capture cannot prove that the repetition of a mutation is safe. +The stable hierarchy primes the next command, so the retry policy does not add another hierarchy read there. + +`waitForAnimationToEnd` uses its own screenshot-stability operation. +It matches upstream's two immediate captures per attempt and the 0.005% normalized absolute RGB-difference threshold. +These captures explicitly bypass ordinary screenshot stabilization. +Thus, the command observes the application and does not recursively wait on another settling policy. +After a successful authored visual wait, screenshot stability discharges the same-generation pending mutation boundary. +No hierarchy capture precedes the visual barrier. +If the visual deadline arrives before a stable pair is observed, that boundary stays pending for the hierarchy settle of the next mutating command. +On iOS, comparison captures use the screenshot surface of the persistent runner. +Thus, both frames come from one warmed transport, and no simulator screenshot setup occurs between polls. + +Upstream Maestro is a version-pinned development reference, not a production dependency. +Conformance is enforced by the three-layer oracle in `scripts/maestro-conformance` (issue #1274). +That oracle replaced the original hand-typed parser fixture. + +1. Layer 1 drives the pinned `maestro-orchestra` YAML parser (`dev.mobile:maestro-orchestra:2.5.1`) over a corpus. + The corpus contains vendored upstream flows plus targeted bug-class flows. + The pinned parser is SHA-verified at regeneration. + Layer 1 captures each parse. + The deterministic verifier replays that capture against our engine. + The verifier classifies every flow as identical / both-reject / we-reject / mismatch. + Each non-identical outcome must be a declared, on-the-record divergence. +2. Layer 2 reads upstream constants (retry cap, animation-wait threshold/timeout, erase cap) straight from the pinned bytecode. + It also reads parser-observed defaults (swipe duration) straight from the pinned bytecode. + It cross-checks each value against the live `MAESTRO_COMPATIBILITY_PRESETS` constant. +3. Layer 3 runs a small set of app-observable differential scenarios through both engines against the real fixture app. + The scenarios cover settle ordering, tap retries, and optional warned-vs-failed. + Layer 3 pins the Maestro CLI to the same version as layers 1-2. + It is scheduled, never per-PR. + +Both generated layers are checked in and verified in CI via `node --test` with no Java. +Per-PR CI cannot re-derive them. +Each fixture therefore carries a `contentHash` seal that the verifier recomputes. +Thus, a hand edit to a captured command or constant fails the build. +The scheduled `conformance-regenerate` job then re-runs the harness against the pinned jars and fails on any byte difference. +That job makes "generated from upstream" enforced rather than documented. + +Layer 3 carries the same declared-divergence contract as layer 1. +A scenario that currently fails declares a `knownDivergence` with a required tracking issue. +The schedule stays green on that known gap, and only undeclared divergences fail. +A declaration that stops reproducing also fails. +Thus, the fix PR must remove the declaration, and the differential becomes the acceptance test for its own findings. +This contract exists because the instrument must not be blocked on the repair of what it just measured. + +Each of the four #1217 regressions has a dedicated fixture: + +- Percent rounding/rejection. +- Target-swipe direction default. +- Retry-cap semantics. +- Settle-loop ordering. + +A pure unit test, rather than a device scenario, pins the runtime half of percent truncation, because a one-pixel delta is not app-observable. +The detector for bug class 4 is an engine-side timing invariant that executes on the device path. +Unit tests exercise that detector against synthetic traces. +On the device path, the detector is currently blocked by a declared `scrollUntilVisible` divergence (#1299). +The differential itself found that divergence on its first working run. + +Documented intentional deviations are encoded as expected-divergence entries rather than silent mismatches: + +- The hierarchy-vs-screenshot animation wait (the constants match, but the mechanism differs). +- The horizontal-swipe in-page presets. +- Condition-poll-as-stabilization. +- Ambiguity strictness under `optional`. +- The omitted iOS 3s pre-tap static gate (`IOSDriver.SCREEN_SETTLE_TIMEOUT_MS`, a layer-2 reference-only vector). ## Performance Contract -The direct engine preserves these properties on Android and iOS across the pager and react-navigation -corpora: - -- total wall time is no slower than the pre-migration compatibility engine; -- an observation immediately after a mutation polls its authored condition directly; a later mutation uses - that result as its stabilization baseline; -- no command captures a second hierarchy merely to re-verify evidence produced within that command; -- absolute coordinate swipes perform one direct viewport query and no accessibility capture; -- mutation-to-mutation stabilization is deferred to the next command boundary and primes the following read; -- percentage swipe conversion preserves authored endpoints exactly; -- helper/runner startup remains amortized across a suite; -- p50/p95 command latency, hierarchy and screenshot captures, tap retries, and transferred hierarchy - bytes are reported separately; -- failure-only diagnostics are excluded from happy-path latency comparisons. - -Android verification must prove the bundled helper backend and version. iOS verification must separate -runner startup from warm command latency. Both platforms rerun the non-Maestro gesture canaries; their -two-pointer plans, executor selection, and app-observable effects must remain unchanged. +The direct engine preserves these properties on Android and iOS across the pager and react-navigation corpora: + +- Total wall time is no slower than the pre-migration compatibility engine. +- An observation immediately after a mutation polls its authored condition directly. + A later mutation uses that result as its stabilization baseline. +- No command captures a second hierarchy only to re-verify evidence produced within that command. +- Absolute coordinate swipes perform one direct viewport query and no accessibility capture. +- Mutation-to-mutation stabilization is deferred to the next command boundary and primes the following read. +- Percentage swipe conversion preserves authored endpoints exactly. +- Helper/runner startup remains amortized across a suite. +- p50/p95 command latency, hierarchy and screenshot captures, tap retries, and transferred hierarchy bytes are reported separately. +- Failure-only diagnostics are excluded from happy-path latency comparisons. + +Android verification must prove the bundled helper backend and version. +iOS verification must separate runner startup from warm command latency. +Both platforms rerun the non-Maestro gesture canaries. +Their two-pointer plans, executor selection, and app-observable effects must remain unchanged. ## Consequences -- agent-device supports selected Maestro Flow syntax and behavior. Unsupported features return - explicit errors, and intentional differences are declared in the conformance fixtures. -- `${...}` interpolation stays variable-lookup-only, by decision (#1292): upstream's GraalJS - expression evaluation is not reimplemented outside `evalScript`, so unresolved or - expression-shaped payloads fail loud with source context; the escape hatches are `evalScript` - (inline expression → `output.*` leaves) and `runScript` (file compute) → `${output.x}` (consume). - `evalScript` is the one command whose payload is evaluated as JavaScript: flow env and prior - `output` leaves are bound as strings in a `node:vm` context and the assigned `output` object - folds back into the flat string-key model, so `${output.uppercaseName}` and - `${output.list.length}` resolve. `assertTrue` (#1295) is scoped to that same lookup-only subset — - literal values and bare `${VAR}` lookups evaluated with a pinned string-truthiness table. - `node:vm` is explicitly not a security sandbox — a bound context's prototype chain still - resolves to host `Object`/`Function`, so an untrusted expression can escape it. `runScript` - already carried this trust assumption for daemon-local flows; `evalScript` inherits the same - assumption but is reachable from a flow accepted over the daemon's remote HTTP surface, so - `MaestroEngineOptions.trustedScripts` gates it — set from `publicNetworkOnly` at the HTTP - boundary (`restrictRemoteHttpRequest`) — and `evalScript` is refused outright when false. -- Shipping two production engines or a runtime fallback between them is rejected because it doubles - semantic and performance ownership. +- agent-device supports selected Maestro Flow syntax and behavior. + Unsupported features return explicit errors. + Intentional differences are declared in the conformance fixtures. +- By decision (#1292), `${...}` interpolation stays variable-lookup-only. + Upstream's GraalJS expression evaluation is not reimplemented outside `evalScript`. + Thus, unresolved or expression-shaped payloads fail loudly with source context. + The escape hatches are `evalScript` (inline expression → `output.*` leaves) and `runScript` (file compute) → `${output.x}` (consume). + `evalScript` is the one command whose payload is evaluated as JavaScript. + Flow env and prior `output` leaves are bound as strings in a `node:vm` context. + The assigned `output` object folds back into the flat string-key model. + Thus, `${output.uppercaseName}` and `${output.list.length}` resolve. + `assertTrue` (#1295) is scoped to that same lookup-only subset. + That subset is literal values and bare `${VAR}` lookups, evaluated with a pinned string-truthiness table. + `node:vm` is explicitly not a security sandbox. + A bound context's prototype chain still resolves to host `Object`/`Function`, so an untrusted expression can escape it. + `runScript` already carried this trust assumption for daemon-local flows. + `evalScript` inherits the same assumption. + However, `evalScript` is reachable from a flow accepted over the daemon's remote HTTP surface. + Thus, `MaestroEngineOptions.trustedScripts` gates `evalScript`. + The HTTP boundary (`restrictRemoteHttpRequest`) sets that option from `publicNetworkOnly`. + If the option is false, `evalScript` is refused outright. +- Shipping two production engines or a runtime fallback between them is rejected because it doubles semantic and performance ownership. - Source provenance and runtime values stay typed through execution. -- Compatibility policy remains local while device behavior stays in shared runtimes and backends. +- Compatibility policy remains local, while device behavior stays in shared runtimes and backends. - Cross-platform correctness may require richer provider query evidence, but not additional round trips. -- ADR 0013 can be rewritten internally without changing Maestro as long as its normalized - single-pointer boundary and executor guarantees remain available. +- ADR 0013 can be rewritten internally without a change to Maestro, as long as its normalized single-pointer boundary and executor guarantees remain available. ## Alternatives Considered -- Embed upstream Orchestra: rejected because Java startup, package weight, driver ownership, and - platform coverage would erase performance and backend advantages. -- Build a shared replay VM first: rejected until native `.ad` needs structured runtime control flow; - one caller does not justify a broader abstraction. -- Keep compiling to typed `SessionAction` variants: rejected because it retains the replay trampoline - and prevents the compatibility engine from owning observation generations directly. +- **Embed upstream Orchestra.** Rejected because Java startup, package weight, driver ownership, and platform coverage would erase performance and backend advantages. +- **Build a shared replay VM first.** Rejected until native `.ad` needs structured runtime control flow. + One caller does not justify a broader abstraction. +- **Keep compiling to typed `SessionAction` variants.** Rejected because it retains the replay trampoline and prevents the compatibility engine from owning observation generations directly. diff --git a/docs/adr/0016-active-session-script-publication.md b/docs/adr/0016-active-session-script-publication.md index 75d30af7fc..e0136ef5b4 100644 --- a/docs/adr/0016-active-session-script-publication.md +++ b/docs/adr/0016-active-session-script-publication.md @@ -2,58 +2,59 @@ ## Status -Accepted; implemented on `main` (`packages/daemon/src/sessions/handlers/session-script-publication.ts`, -`packages/daemon/src/core/session-script-active-publication.ts`), including the #1349 identity-level -destination-guard amendment. +Accepted. +This ADR is implemented on `main`, including the #1349 identity-level destination-guard amendment. + +Implementation owners are `packages/daemon/src/sessions/handlers/session-script-publication.ts` and `packages/daemon/src/core/session-script-active-publication.ts`. ## Rules at a glance -Normative summary; the binding contracts and refusal cases are in [Decision](#decision). - -- `session save-script [path] [--force]` publishes the current ordinary script recording without - closing the app or the session; the session stays live at the destination. -- Recording lifecycle: **ARMED** (only by a new session's initial `open --save-script`) -> - **ABORTED** (a second successful plain `open`) or **PUBLISHED** (successful save). ABORTED and - PUBLISHED are terminal until the session is destroyed; a filesystem/target failure leaves the - recording ARMED and retryable. -- Publication requires exactly one initial recorded `open`, a portable selector-`wait` destination - guard after the last descriptor-classified mutating action — since #1349 the guard's `target-v1` - annotation must be `verification: "verified"` — and no unresolved session-local `@ref`. Every - refusal happens before filesystem work and names its recovery. -- Repair transactions (ADR 0012) are disjoint: a session with `saveScriptBoundary` set refuses this - action without changing repair state. -- `ReplayCommandResult.sessionActive` (from the daemon's session store, never script re-parsing) is - what lets a one-shot client keep the daemon alive on a close-less handoff. -- Sensitive `fill` inputs must be recorded as placeholders via `fill --record-as ` - (ADR 0017, shipped for #1348); unparameterized `fill`/`type` values persist literally into the - artifact, so a secret entered without `--record-as` is published. The protection is - recording-session-scoped (ADR 0017's #1398 amendment): a later action's own recorded evidence can - never re-serialize an app-rendered echo of an already-parameterized value either. +This section is a normative summary. +[Decision](#decision) contains the binding contracts and refusal cases. + +- `session save-script [path] [--force]` publishes the current ordinary script recording without closing the app or the session. + The session stays live at the destination. +- Recording lifecycle: + - Only the initial `open --save-script` of a new session establishes **ARMED**. + - From ARMED, a second successful plain `open` causes **ABORTED**. + - From ARMED, only a successful atomic `session save-script` establishes **PUBLISHED**. + - ABORTED and PUBLISHED are terminal until the session is destroyed. + - A filesystem or target failure leaves the recording ARMED and retryable. +- Publication requires exactly one initial recorded `open`, a portable selector-`wait` destination guard after the last descriptor-classified mutating action, and no unresolved session-local `@ref`. + Since #1349, the `target-v1` annotation of the guard must be `verification: "verified"`. + Every refusal happens before filesystem work and names its recovery. +- Repair transactions (ADR 0012) are disjoint from this lifecycle. + A session with `saveScriptBoundary` set refuses this action without a change to repair state. +- `ReplayCommandResult.sessionActive` comes from the daemon's session store, never from a re-parse of the script. + This signal lets a one-shot client keep the daemon alive on a close-less handoff. +- Sensitive `fill` inputs must be recorded as placeholders through `fill --record-as ` (ADR 0017, shipped for #1348). + Unparameterized `fill`/`type` values persist literally into the artifact, so a secret entered without `--record-as` is published. + The protection is scoped to the recording session (ADR 0017's #1398 amendment). + A later action's own recorded evidence can never re-serialize an app-rendered echo of an already-parameterized value either. ## Context -The replay repair study found re-recording a drifted journey cheaper than repairing it in two of the -three measured drift classes. The immediate reusable unit is an **open-to-destination script**: a -self-contained `.ad` script that opens the app, performs the complete journey to screen X, verifies a -destination landmark, and leaves the app session active there so an agent can continue with new work. - -Ordinary script recording currently combines two concerns. `open --save-script[=]` arms -recording before the first interaction so selector chains and ADR 0012 `target-v1` identity evidence -are captured at action time. `close` publishes the accumulated script, records a terminal `close`, and -tears down the session. That artifact is suitable as a closed test flow, but not as a live starting -state. - -Publishing only at the end of an ordinary unarmed session is insufficient. Session history retains the -commands, but the resolved target tree needed for `target-v1` evidence is deliberately discarded after -each action. Late publication can serialize selector text; it cannot reconstruct which element was -actually acted on. Making identity capture unconditional for every session would change normal -interaction execution: recording disables direct selector fast paths when a capture-backed route is -required for evidence. That performance and data-retention change has not been measured. - -Issue [#1346](https://github.com/callstack/agent-device/issues/1346) and the throwaway -[`session-save-script` prototype](https://github.com/callstack/agent-device/tree/agent/prototype-session-save-replay/scripts/prototypes/session-save-script) -record the motivating workflow and API trial. Composable lifecycle-free fragments remain a separate -decision in [#1336](https://github.com/callstack/agent-device/issues/1336). +The replay repair study found that re-recording a drifted journey was cheaper than repairing it in two of the three measured drift classes. +The immediate reusable unit is an **open-to-destination script**. +This self-contained `.ad` script opens the app, performs the complete journey to screen X, and verifies a destination landmark. +It leaves the app session active there, so an agent can continue with new work. + +Ordinary script recording currently combines two concerns. +`open --save-script[=]` arms recording before the first interaction. +Thus, selector chains and ADR 0012 `target-v1` identity evidence are captured at action time. +`close` publishes the accumulated script, records a terminal `close`, and tears down the session. +That artifact is suitable as a closed test flow, but not as a live starting state. + +Publication only at the end of an ordinary unarmed session is insufficient. +Session history retains the commands. +But the resolved target tree that `target-v1` evidence needs is deliberately discarded after each action. +Late publication can serialize selector text, but it cannot reconstruct which element the action actually targeted. +Unconditional identity capture for every session would change normal interaction execution. +Recording disables direct selector fast paths when evidence requires a capture-backed route. +That performance and data-retention change has not been measured. + +Issue [#1346](https://github.com/callstack/agent-device/issues/1346) and the disposable [`session-save-script` prototype](https://github.com/callstack/agent-device/tree/agent/prototype-session-save-replay/scripts/prototypes/session-save-script) record the motivating workflow and API trial. +Composable lifecycle-free fragments remain a separate decision in [#1336](https://github.com/callstack/agent-device/issues/1336). ## Decision @@ -66,290 +67,310 @@ agent-device wait 'role="heading" label="Screen X"' agent-device session save-script ``` -`session save-script [path] [--force]` publishes the current ordinary script recording without -closing the app or deleting the session. An explicit `path` retargets the recording using the existing -target/force authorization rules. Without it, publication uses the path armed by `open --save-script` -or the existing generated default. +`session save-script [path] [--force]` publishes the current ordinary script recording without closing the app or deleting the session. +An explicit `path` retargets the recording under the existing target/force authorization rules. +Without it, publication uses the path armed by `open --save-script` or the existing generated default. ### Recording lifecycle An ordinary recording eligible for active-session publication has three states: -- **ARMED**: established only when a new session's initial successful - `open --save-script[=]` is recorded as action zero. The session records portable action inputs - and fresh target identity evidence. -- **ABORTED**: reached when another plain `open` succeeds while ARMED. The app operation may continue, - but the recording is no longer a single-open bootstrap and close-time publication is disarmed. The - successful `open` response warns that a fresh session is required to author another script. -- **PUBLISHED**: reached only after `session save-script` atomically commits the complete history from - the sole recorded `open` through the current action. The session remains active at the destination, - but close-time script publication is disarmed. - -A filesystem or target-collision failure leaves the recording ARMED, including its path and same-target -`--force` authorization, so the caller can correct the target or permissions and retry. ABORTED and -PUBLISHED are terminal until the session is destroyed. `session save-script` in either state fails -loudly and never writes. - -Every existing arming entry point respects terminality. `open --save-script` on any existing session — -unarmed, ARMED, ABORTED, or PUBLISHED — is rejected before app dispatch; a plain later `open` is allowed, -causing ARMED to become ABORTED while leaving ABORTED/PUBLISHED unchanged. -`close --save-script[=]` in ABORTED or PUBLISHED is rejected before platform close or filesystem -work, so the caller can retry with plain `close`. Plain `close` tears down ABORTED/PUBLISHED without -writing; closing an unpublished ARMED recording retains the existing close-time publication behavior. A -fresh session is the only re-arming boundary. - -> **Amendment (2026-08-02, shipped).** A session that was never armed — no recorded -> `open --save-script` at all — has no fourth pre-ARMED state name here, but it previously fell -> through the same close-time write path as an ARMED recording: `close --save-script` on it folded -> the request into the authoring lifecycle at record time and published anyway. Live evidence -> showed this produces a script whose actions carry selector fallback chains but no recording-time -> `target-v1` evidence, with no signal to the caller that evidence capture never ran — degraded -> replay verification is worse than a loud refusal. `close --save-script` on a never-armed session -> is now rejected before any teardown or filesystem work, the same way ABORTED/PUBLISHED are, -> naming `open --save-script` as the recovery; a plain `close` still tears the session down -> without writing. This is distinct from -> [#1533](https://github.com/callstack/agent-device/issues/1533), which is about an -> already-ARMED-then-ABORTED session whose flag ingress re-enabled recording and let a -> *bare* `close` (no `--save-script` on the close itself) publish; that case is resolved by the -> amendments below. - -> **Amendment (#1533, shipped).** ABORTED terminality above was enforced only by -> `abortAuthoringOnSecondOpen` clearing `session.recordSession` — inert by ordering, not by -> construction. `recordSession` is an evidence-capture flag that several surfaces set directly, so -> a later `--save-script` re-armed it while the status stayed ABORTED, and a *bare* `close` then -> published the full session log through a writer that only knew how to refuse repair -> transactions. The refusal this ADR specifies for `close --save-script` in ABORTED promises the -> caller that plain `close` "tears down without writing", so the gap also made an existing error -> message untrue. -> -> ABORTED is now terminal by construction. `--save-script` arms recording through one rule owned by -> the publication projection, which answers "not recording" for an ABORTED lifecycle on every -> surface that handles the flag — the re-open builder, the close finalizer, and the recorded-action -> ingress — so the flag can no longer contradict the status. Publication authorization is likewise -> the aggregate's to answer: the writer asks one publication-blocked question covering -> not-recording, an uncommittable or committed repair, and an ABORTED authoring lifecycle alike, so -> every path that reaches it (bare `close`, teardown, idle-reap, active publication) refuses. ARMED -> and PUBLISHED lifecycles and every repair transaction are unchanged; no state name, transition, -> or entry point in the lifecycle above is added or altered. - -> **Amendment (#1533 follow-up, shipped).** The rule above still relied on a stored -> `SessionState.recordSession` that every writer had to set in step with the lifecycle. Routing the -> writes through one place made the two agree; it did not make disagreement unrepresentable, and -> the field remained a second source of truth for a question the aggregate already answered. -> -> Recording is now **derived**, not stored. `isRecordingPublication` reads it off the lifecycle — -> ordinary authoring records only while ARMED; a repair transaction records for its whole lifetime, -> terminal statuses included — and the field is gone. Three consequences worth stating, because -> they are what the derivation buys: -> -> - No surface can arm recording without moving the lifecycle that authorizes it, so #1533's drift -> is unrepresentable rather than guarded against. `buildNextOpenSession` and the close finalizer -> make no recording decision at all now. -> - The writer's publication gate is answered entirely by the aggregate. Its separate ABORTED check -> is gone, because a terminal authoring lifecycle is already not recording — no second gate that -> could disagree with the first about *authoring*. -> - Evidence capture and publication authorization now derive from the same aggregate, but they -> remain **distinct predicates**, and deliberately so. They coincide for ordinary authoring: -> ARMED both records and publishes, ABORTED and PUBLISHED do neither. They do NOT coincide for -> repair — `isRecordingPublication` is true for every repair status, `committed` and `aborted` -> included, while publication additionally requires `isRepairArmedWriteBlocked` to pass, which -> refuses a committed transaction (idempotent no-op) and one that is not yet committable (ADR -> 0012 decision 6, C2). A repair that captures evidence is therefore not necessarily a repair -> that may publish, and collapsing the two would silently republish or commit a prefix. -> -> This is behavior-preserving: the derivation reproduces exactly what the flag held at every -> transition. Whether a *committed* repair should still capture recording-time evidence is a real -> question this deliberately does not answer — the old flag said yes, and changing that is a -> behavior change, not a derivation. - -This lifecycle is distinct from ADR 0012's repair transaction. `session save-script` rejects a session -with `saveScriptBoundary` set and directs the caller to finish or abort the repair through its existing -`replay --from` and teardown commit protocol. Active-session publication never marks a repair COMPLETE, -commits a healed slice, writes `# agent-device:heal-complete`, or changes repair tombstone semantics. +- **ARMED**: only the initial successful `open --save-script[=]` of a new session, recorded as action zero, establishes this state. + The session records portable action inputs and fresh target identity evidence. +- **ABORTED**: another successful plain `open` while ARMED establishes this state. + The app operation may continue. + But the recording is no longer a single-open bootstrap, and close-time publication is disarmed. + The successful `open` response warns that a fresh session is required to author another script. +- **PUBLISHED**: only a successful atomic `session save-script` establishes this state. + That command commits the complete history from the sole recorded `open` through the current action. + The session remains active at the destination, but close-time script publication is disarmed. + +A filesystem or target-collision failure leaves the recording ARMED, including its path and same-target `--force` authorization. +Thus, the caller can correct the target or permissions and retry. +ABORTED and PUBLISHED are terminal until the session is destroyed. +In either state, `session save-script` fails with an explicit error and never writes. + +Every existing arming entry point respects terminality. +On any existing session, `open --save-script` is rejected before app dispatch. +This includes unarmed, ARMED, ABORTED, and PUBLISHED sessions. +A later plain `open` is allowed. +It changes ARMED to ABORTED and leaves ABORTED/PUBLISHED unchanged. +In ABORTED or PUBLISHED, `close --save-script[=]` is rejected before platform close or filesystem work. +Thus, the caller can retry with plain `close`. +Plain `close` tears down ABORTED/PUBLISHED without writing. +The close of an unpublished ARMED recording retains the existing close-time publication behavior. +A fresh session is the only re-arming boundary. + +**Amendment (2026-08-02, shipped).** A never-armed session has no recorded `open --save-script` at all. +The lifecycle above has no fourth pre-ARMED state name for it. +Before this amendment, such a session used the same close-time write path as an ARMED recording. +`close --save-script` on it added the request to the authoring lifecycle at record time and published anyway. +Live evidence showed that this produced a script whose actions had selector fallback chains but no recording-time `target-v1` evidence. +The caller got no signal that evidence capture never ran. +Degraded replay verification is worse than an explicit refusal. +Now, `close --save-script` on a never-armed session is rejected before any teardown or filesystem work, the same way as in ABORTED/PUBLISHED. +The refusal names `open --save-script` as the recovery. +A plain `close` still tears the session down without writing. +This case is distinct from [#1533](https://github.com/callstack/agent-device/issues/1533), which concerns an already-ARMED-then-ABORTED session. +In that session, flag ingress re-enabled recording and let a *bare* `close` publish. +A bare `close` has no `--save-script` on the close itself. +The amendments below resolve that case. + +**Amendment (#1533, shipped).** Before this amendment, only `abortAuthoringOnSecondOpen` enforced the ABORTED terminality above. +It cleared `session.recordSession`. +That enforcement depended on operation order, not on construction. +`recordSession` was an evidence-capture flag that several surfaces set directly. +Thus, a later `--save-script` re-armed it while the status stayed ABORTED. +A *bare* `close` then published the full session log through a writer that could refuse only repair transactions. +For `close --save-script` in ABORTED, this ADR specifies a refusal that promises the caller that plain `close` "tears down without writing". +Thus, the gap also made an existing error message untrue. + +ABORTED is now terminal by construction. +`--save-script` arms recording through one rule that the publication projection owns. +That rule answers "not recording" for an ABORTED lifecycle on every surface that handles the flag. +These surfaces are the re-open builder, the close finalizer, and the recorded-action ingress. +Thus, the flag can no longer contradict the status. +The aggregate likewise answers publication authorization. +The writer asks one publication-blocked question. +That question covers not-recording, an uncommittable or committed repair, and an ABORTED authoring lifecycle alike. +Thus, every path that reaches the writer refuses: bare `close`, teardown, idle-reap, and active publication. +ARMED and PUBLISHED lifecycles and every repair transaction are unchanged. +No state name, transition, or entry point in the lifecycle above is added or altered. + +**Amendment (#1533 follow-up, shipped).** The rule above still relied on a stored `SessionState.recordSession` field. +Every writer had to set that field in step with the lifecycle. +Routing the writes through one place made the two agree, but it did not make disagreement unrepresentable. +The field remained a second source of truth for a question that the aggregate already answered. + +Recording is now **derived**, not stored. +`isRecordingPublication` reads it from the lifecycle, and the field is gone: + +- Ordinary authoring records only while ARMED. +- A repair transaction records for its whole lifetime, terminal statuses included. + +The derivation has three consequences: + +- No surface can arm recording without a move of the lifecycle that authorizes it. + Thus, the #1533 drift is unrepresentable, not only guarded against. + `buildNextOpenSession` and the close finalizer now make no recording decision at all. +- The aggregate answers the whole publication gate of the writer. + The separate ABORTED check of the writer is gone, because a terminal authoring lifecycle is already not recording. + Thus, no second gate can disagree with the first about *authoring*. +- Evidence capture and publication authorization now derive from the same aggregate. + But they deliberately remain **distinct predicates**. + - They coincide for ordinary authoring: ARMED both records and publishes, and ABORTED and PUBLISHED do neither. + - They do NOT coincide for repair. + `isRecordingPublication` is true for every repair status, `committed` and `aborted` included. + Publication additionally requires `isRepairArmedWriteBlocked` to pass. + That check refuses a committed transaction (idempotent no-op) and a transaction that is not yet committable (ADR 0012 decision 6, C2). + - Thus, a repair that captures evidence is not necessarily a repair that may publish. + A merge of the two predicates would silently republish or commit a prefix. + +This change is behavior-preserving. +The derivation reproduces exactly what the flag held at every transition. +This amendment deliberately does not answer one real question: should a *committed* repair still capture recording-time evidence? +The old flag said yes. +A change to that answer is a behavior change, not a derivation. + +This lifecycle is distinct from the ADR 0012 repair transaction. +`session save-script` rejects a session with `saveScriptBoundary` set. +The refusal directs the caller to finish or abort the repair through its existing `replay --from` and teardown commit protocol. +Active-session publication never: + +- Marks a repair COMPLETE. +- Commits a healed slice. +- Writes `# agent-device:heal-complete`. +- Changes repair tombstone semantics. ### Destination readiness and replay handoff -The destination is an authored postcondition, not the last navigation action. Before publication, the -recorded suffix after the last mutating action must contain a `wait` whose target is a portable selector -or selector chain identifying a landmark on the ready destination screen. A duration wait, `wait stable`, -or `wait @ref` does not qualify, though `wait stable` may follow the landmark wait. The publisher validates -the serialized guard before filesystem work and never relies on repair-only bare-ref rejection. -`session save-script` refuses publication without this **destination guard** and tells the author to -record one. V1 does not infer a screen identity from a snapshot or synthesize an implicit guard. - -> **Amendment (#1349, shipped).** The guard is now identity-level, not merely selector-level. A -> selector wait recorded while ARMED captures landmark-mode `target-v1` evidence for the matched -> element (ADR 0012's read-only-step amendment), and a qualifying destination guard is exactly a -> selector wait whose annotation is `verification: "verified"` — a selector wait on an identity-empty -> element (no id and no label) records no annotation and does not qualify, so publication refuses it -> with a recovery hint naming a labeled/id-bearing landmark. On replay, polling is preserved: the wait -> keeps polling until a selector match carries the recorded identity (local identity + leaf-anchored -> ancestry prefix), and a deadline reached with only same-selector impostors fails closed as an -> `identity-mismatch` `REPLAY_DIVERGENCE` before the wait reports success. The reshuffled-screen -> false-pass below is covered by a provider-scenario regression (record → publish → replay against a -> reshuffled tree whose same-label node sits under a different id/ancestry). -> -> **Amendment (#1398, ADR 0017).** An identity-empty landmark is not the only case that fails to -> qualify: a selector wait whose only identity is an app-rendered echo of a literal the recording -> session already parameterized via `fill --record-as` also records no annotation (ADR 0017's -> session-scoped echo protection), so it does not qualify as a destination guard either. Publication -> refuses it with the same recovery hint, directing the author to a stable, non-value-bearing landmark -> — an enforced version of exactly the fix issue #1398's motivating scenario applied by hand. - -The original selector-level caveat, retained as context: the guard proved that an element matching its -selector exists, not that it is the same landmark element observed while authoring, so a reshuffled -screen containing the same weak label elsewhere could false-pass. -[#1349](https://github.com/callstack/agent-device/issues/1349) owned the identity design for waits and -remaining read-only steps. It preserves polling: ADR 0012's pre-action `target-v1` -verification cannot be attached to a wait unchanged because a not-yet-present landmark is the expected -starting condition, so the identity check runs after the wait's selector resolves and before the wait -reports success (see ADR 0012's #1349 amendment for the mechanism and the read-only-step coverage -classification). - -The phrase "last mutating action" is derived from a request-sensitive recording-effect trait on the -central `CommandDescriptor`, required for every command that records session actions and guarded by a -completeness test. It is not a publisher-local command-name set. The trait distinguishes app-state -mutation from observation for subcommands such as read-only versus mutating `find`, `keyboard`, and -`alert` actions; the existing conservative `refFrameEffect: may-invalidate` classification is not precise -enough for this boundary. - -On consumption, a script without `close` preserves the existing replay behavior: the named session stays -active and the successful `ReplayCommandResult` returns its `session` id. The caller binds subsequent -commands to that returned id. Replay reports success only after the destination guard completes; the -absence of `close` changes neither action dispatch nor when replay reports success. - -**Response shape (issue #1384).** `ReplayCommandResult` carries a required `sessionActive: boolean`, -computed from whether `session` still exists in the daemon's own store when the response is built — never -by re-parsing the script for `close`. This is what lets the real CLI/IPC client (not just the in-process -handler) actually honor "the session stays active": without an explicit signal in the response, the -client's one-shot `replay`/`test` teardown had no way to distinguish a still-active handoff from a -finished one, and tore the owning daemon down regardless (`src/daemon-client/daemon-client-lifecycle.ts`). -`sessionActive` is `true` for every close-less run (including a `--from` resume) and `false` once the -script's terminal `close` — or a repair-armed run's deferred equivalent — has executed. +The destination is an authored postcondition, not the last navigation action. +Before publication, the recorded suffix after the last mutating action must contain a `wait` whose target is a portable selector or selector chain. +That selector must identify a landmark on the ready destination screen. +A duration wait, `wait stable`, or `wait @ref` does not qualify. +But `wait stable` may follow the landmark wait. +The publisher validates the serialized guard before filesystem work. +The publisher never relies on repair-only bare-ref rejection. +Without this **destination guard**, `session save-script` refuses publication and tells the author to record one. +V1 does not infer a screen identity from a snapshot or synthesize an implicit guard. + +**Amendment (#1349, shipped).** The guard is now identity-level, not merely selector-level. +A selector wait recorded while ARMED captures landmark-mode `target-v1` evidence for the matched element (ADR 0012's read-only-step amendment). +A qualifying destination guard is exactly a selector wait whose annotation is `verification: "verified"`. +A selector wait on an identity-empty element (no id and no label) records no annotation and does not qualify. +Publication refuses it with a recovery hint that names a labeled/id-bearing landmark. + +On replay, polling is preserved. +The wait keeps polling until a selector match carries the recorded identity (local identity + leaf-anchored ancestry prefix). +If the wait reaches its deadline with only same-selector impostors, it fails closed as an `identity-mismatch` `REPLAY_DIVERGENCE` before the wait reports success. +A provider-scenario regression covers the reshuffled-screen false-pass below. +That regression records, publishes, then replays against a reshuffled tree whose same-label node is under a different id/ancestry. + +**Amendment (#1398, ADR 0017).** An identity-empty landmark is not the only case that fails to qualify. +A selector wait can have an app-rendered echo as its only identity. +If that echo is of a literal that the recording session already parameterized through `fill --record-as`, the wait also records no annotation. +This is ADR 0017's session-scoped echo protection. +Thus, that wait does not qualify as a destination guard either. +Publication refuses it with the same recovery hint and directs the author to a stable, non-value-bearing landmark. +This refusal enforces exactly the fix that the motivating scenario of issue #1398 applied by hand. + +The original selector-level caveat is retained as context. +The guard proved that an element that matches its selector exists. +It did not prove that the element is the same landmark element observed while authoring. +Thus, a reshuffled screen with the same weak label elsewhere could false-pass. +[#1349](https://github.com/callstack/agent-device/issues/1349) owned the identity design for waits and remaining read-only steps. +That design preserves polling. +ADR 0012's pre-action `target-v1` verification cannot be attached to a wait unchanged, because a not-yet-present landmark is the expected starting condition. +Thus, the identity check runs after the wait's selector resolves and before the wait reports success. +ADR 0012's #1349 amendment defines the mechanism and the read-only-step coverage classification. + +The phrase "last mutating action" derives from a request-sensitive recording-effect trait on the central `CommandDescriptor`. +Every command that records session actions requires this trait, and a completeness test guards it. +It is not a publisher-local command-name set. +The trait distinguishes app-state mutation from observation for subcommands, such as read-only versus mutating `find`, `keyboard`, and `alert` actions. +The existing conservative `refFrameEffect: may-invalidate` classification is not precise enough for this boundary. + +On consumption, a script without `close` preserves the existing replay behavior. +The named session stays active, and the successful `ReplayCommandResult` returns its `session` id. +The caller binds subsequent commands to that returned id. +Replay reports success only after the destination guard completes. +The absence of `close` changes neither action dispatch nor when replay reports success. + +**Response shape (issue #1384).** `ReplayCommandResult` carries a required `sessionActive: boolean`. +The daemon computes it when it builds the response, from whether `session` still exists in the daemon's own store. +The daemon never computes it by a re-parse of the script for `close`. +This field lets the real CLI/IPC client (not only the in-process handler) actually honor "the session stays active". +Before this field, the response had no explicit signal. +The one-shot `replay`/`test` teardown of the client had no way to distinguish a still-active handoff from a finished one. +It tore the owning daemon down regardless (`src/daemon-client/daemon-client-lifecycle.ts`). +`sessionActive` is `true` for every close-less run, including a `--from` resume. +It is `false` once the terminal `close` of the script, or the deferred equivalent of a repair-armed run, has executed. ### Sensitive inputs -Executable `.ad` artifacts serialize ordinary action inputs literally; diagnostic and event-log -redaction cannot protect an input that replay must later execute. ADR 0017 resolves #1348 for explicit -sensitive fills: `fill ... --record-as PASSWORD` sends the live text to the app while recording only -`${PASSWORD}`. Replay receives the value through `AD_VAR_PASSWORD` or `--env PASSWORD=`. +Executable `.ad` artifacts serialize ordinary action inputs literally. +Diagnostic and event-log redaction cannot protect an input that replay must later execute. +ADR 0017 resolves #1348 for explicit sensitive fills. +`fill ... --record-as PASSWORD` sends the live text to the app while it records only `${PASSWORD}`. +Replay receives the value through `AD_VAR_PASSWORD` or `--env PASSWORD=`. -This is opt-in and fill-only. Unparameterized fill/type inputs remain literal artifact content, so -authors must use ADR 0017 for each sensitive fill and avoid secret-bearing `type` steps. CLI help states -both the safe workflow and the remaining literal-input warning next to publication guidance. +This protection is opt-in and fill-only. +Unparameterized fill/type inputs remain literal artifact content. +Thus, authors must use ADR 0017 for each sensitive fill and avoid secret-bearing `type` steps. +CLI help states both the safe workflow and the remaining literal-input warning next to publication guidance. -ADR 0017's #1398 amendment extends this protection to the whole recording session: a later, unrelated -recorded action can no longer re-serialize an app-rendered echo of an already-parameterized literal into -its own result or `target-v1` evidence, so a value protected once by `--record-as` stays protected for -every action recorded after it in the same session, not only its own originating fill. +ADR 0017's #1398 amendment extends this protection to the whole recording session. +A later, unrelated recorded action can no longer re-serialize an app-rendered echo of an already-parameterized literal into its own result or `target-v1` evidence. +Thus, a value protected once by `--record-as` stays protected for every action recorded after it in the same session, not only in its own originating fill. ### Artifact contract The published `.ad`: -- contains exactly one recorded `open` as its first action and every recordable action through the - publication request; -- contains a portable selector/selector-chain destination guard after its last descriptor-classified - mutating action; -- does not append or serialize `session save-script` or `close`; -- uses the ordinary session context header, selector-chain optimization, and canonical `target-v1` - annotations captured while ARMED; -- fails loudly rather than emitting an unresolved session-local `@ref` or dropping target evidence that - ADR 0012 requires for an element-targeting recorded action; and -- uses the existing same-directory atomic publication primitive, refusing every existing target unless - `--force` authorizes atomic replacement. +- Contains exactly one recorded `open` as its first action, and every recordable action through the publication request. +- Contains a portable selector/selector-chain destination guard after its last descriptor-classified mutating action. +- Does not append or serialize `session save-script` or `close`. +- Uses the ordinary session context header, selector-chain optimization, and canonical `target-v1` annotations captured while ARMED. +- Fails with an explicit error instead of emitting an unresolved session-local `@ref`. + It also fails with an explicit error instead of dropping target evidence that ADR 0012 requires for an element-targeting recorded action. +- Uses the existing same-directory atomic publication primitive. + It refuses every existing target unless `--force` authorizes atomic replacement. The success response identifies the final path and session and reports the number of serialized actions. -The command must fail before writing when there is no active session, recording was not armed, the -recording is ABORTED or PUBLISHED, the history does not contain exactly one initial `open`, no portable -destination guard exists, or a repair transaction owns the session. Every failure explains the recovery -action; none degrades to `{ written: false }` success. +The command must fail before writing in each of these cases: + +- There is no active session. +- Recording was not armed. +- The recording is ABORTED or PUBLISHED. +- The history does not contain exactly one initial `open`. +- No portable destination guard exists. +- A repair transaction owns the session. + +Every failure explains the recovery action. +No failure degrades to `{ written: false }` success. ### Surface and naming -V1 extends the existing `session` command and typed session client surface. It does not introduce -`script start/stop`, marks, or a second replay engine. CLI help makes the two phases explicit: the -existing `--save-script` flag arms evidence capture, while `session save-script` publishes without -teardown. +V1 extends the existing `session` command and the typed session client surface. +It does not introduce `script start/stop`, marks, or a second replay engine. +CLI help makes the two phases explicit: + +1. The existing `--save-script` flag arms evidence capture. +2. `session save-script` publishes without teardown. -This ADR does not rename or deprecate `--save-script`. The flag and session action name the same persisted -artifact: `open --save-script` configures the armed recording's eventual target, while -`session save-script` requests publication now. `save-replay` is rejected because replay is the act of -executing that script, not the artifact being saved. +This ADR does not rename or deprecate `--save-script`. +The flag and the session action name the same persisted artifact. +`open --save-script` configures the eventual target of the armed recording. +`session save-script` requests publication now. +`save-replay` is rejected, because replay is the act of executing that script, not the artifact being saved. ## Consequences -- Agents can record onboarding or deep navigation as one self-contained starting state, replay it from - scratch, and continue from the resulting live session. -- The workflow has two explicit moments because evidence must be armed before the first target action and - the destination is known only when the caller publishes. +- Agents can record onboarding or deep navigation as one self-contained starting state, replay it from scratch, and continue from the resulting live session. +- The workflow has two explicit moments. + Evidence must be armed before the first target action. + The destination is known only when the caller publishes. - Normal unarmed interactions keep their current fast paths and retention behavior. - A successful active-session publication cannot collide with a later close-time auto-save. -- A second successful open abandons the in-flight artifact instead of silently publishing a multi-open - bootstrap; authoring resumes only in a fresh session. -- Intermediate lifecycle-free fragments, entry guards, include semantics, composed digests, and shared - fragment pinning remain entirely under #1336. -- Secret-bearing fill authoring uses ADR 0017's explicit `--record-as ` contract. Unparameterized - fill/type inputs remain literal script content. Arbitrary history ranges remain out of scope. -- On consumption (issue #1384), a `replay` whose script has no terminal `close` leaves a live daemon and - app session behind — including the ordinary one-shot `replay` invocation's own owned/ephemeral daemon, - which the client keeps alive and reports a `--state-dir` address hint for instead of tearing down. An - unattended close-less replay (e.g. in CI) therefore leaks a session exactly like one opened - interactively: bounded by ordinary idle-reap or an explicit `close`, never by the one-shot command's own - process lifetime. This is the intended shape of "the caller owns close," not an accident, and recorded - test flows already end with `close` so `test` runs are unaffected. +- A second successful open abandons the in-flight artifact instead of silently publishing a multi-open bootstrap. + Authoring resumes only in a fresh session. +- Intermediate lifecycle-free fragments, entry guards, include semantics, composed digests, and shared fragment pinning remain entirely under #1336. +- Secret-bearing fill authoring uses ADR 0017's explicit `--record-as ` contract. + Unparameterized fill/type inputs remain literal script content. + Arbitrary history ranges remain out of scope. +- On consumption (issue #1384), a `replay` whose script has no terminal `close` leaves a live daemon and app session behind. + This includes the owned/ephemeral daemon of the ordinary one-shot `replay` invocation. + The client keeps that daemon alive and reports a `--state-dir` address hint for it instead of a teardown. + Thus, an unattended close-less replay (for example, in CI) leaks a session exactly like a session opened interactively. + Ordinary idle-reap or an explicit `close` bounds it, never the process lifetime of the one-shot command. + This is the intended shape of "the caller owns close", not an accident. + Recorded test flows already end with `close`, so `test` runs are unaffected. ## Alternatives Considered -- **Save any session history at the end:** rejected because target identity evidence cannot be - reconstructed after the interaction and the resulting artifact would undercut ADR 0012's provenance - model. -- **Capture full target evidence in every session:** rejected until its direct-path latency, capture - count, memory, and event-log costs are measured. It would alter ordinary interaction behavior to make - one authoring command shorter. -- **`close --no-close` or `--save-script --no-close`:** rejected because a command named for teardown - would conditionally preserve the session and because it would not solve late arming. -- **General `script start/stop` or history marks:** rejected because the accepted v1 boundary is exactly - one recorded `open` through one destination. Arbitrary slices require entry-state semantics and belong - with fragment design. -- **`replay save`:** rejected because `replay ` consumes an artifact while publication consumes a - live session; the session owns the source data and lifecycle. -- **Infer a destination fingerprint at publication:** rejected for v1 because screen identity and - readiness are app semantics. A caller-authored target wait is explicit, already recordable, and fails - at the correct point during cold replay. +- **Save any session history at the end.** Rejected, because target identity evidence cannot be reconstructed after the interaction. + The resulting artifact would weaken ADR 0012's provenance model. +- **Capture full target evidence in every session.** Rejected until its direct-path latency, capture count, memory, and event-log costs are measured. + It would alter ordinary interaction behavior to make one authoring command shorter. +- **`close --no-close` or `--save-script --no-close`.** Rejected, because a command named for teardown would conditionally preserve the session. + It also would not solve late arming. +- **General `script start/stop` or history marks.** Rejected, because the accepted v1 boundary is exactly one recorded `open` through one destination. + Arbitrary slices require entry-state semantics and belong with fragment design. +- **`replay save`.** Rejected, because `replay ` consumes an artifact while publication consumes a live session. + The session owns the source data and lifecycle. +- **Infer a destination fingerprint at publication.** Rejected for v1, because screen identity and readiness are app semantics. + A caller-authored target wait is explicit, already recordable, and fails at the correct point during cold replay. ## Validation Required for Implementation -- An unarmed session refuses publication before filesystem work and names `open --save-script` as the - recovery. -- An armed session without a destination guard refuses publication before filesystem work and names a - selector-targeted `wait` as the recovery; `wait @ref`, duration waits, and `wait stable` are covered - refusal cases. -- A second successful plain `open` transitions ARMED to ABORTED and disables all publication, while +- An unarmed session refuses publication before filesystem work and names `open --save-script` as the recovery. +- An armed session without a destination guard refuses publication before filesystem work and names a selector-targeted `wait` as the recovery. + `wait @ref`, duration waits, and `wait stable` are covered refusal cases. +- A second successful plain `open` changes ARMED to ABORTED and disables all publication. `open --save-script` on an existing session is rejected before app dispatch. -- An armed session publishes `open` plus target-annotated actions without `close`, returns the final path, - remains active, and can continue accepting commands. -- The artifact replays from a cold start, completes its destination guard, returns the live session id, - and accepts a subsequent command on that session. -- Every action in ADR 0012's existing target-binding command set has canonical identity evidence and no - unresolved `@ref` reaches disk; since #1349 the destination guard additionally requires verified - recorded landmark identity, with a reshuffled-screen false-pass regression proving replay fails - closed. -- Existing-target refusal preserves the original bytes; `--force` replaces atomically; a failed publish - remains retryable. -- After PUBLISHED, later ordinary actions remain usable, repeated `session save-script` fails, plain - `open --relaunch` cannot re-arm, and `open --save-script` is rejected before app dispatch. -- In ABORTED/PUBLISHED, `close --save-script[=]` is rejected before platform close and plain - `close` tears down without writing; closing an unpublished ARMED recording preserves current - close-time publication behavior. -- On a never-armed session (2026-08-02 amendment), `close --save-script` is likewise rejected before - platform close or filesystem work, naming `open --save-script` as the recovery; plain `close` still - tears down without writing, and the session is not deleted by the rejected request. -- Descriptor completeness tests classify every recordable request's mutation effect, including - request-sensitive read-only/mutating subcommands, and destination-guard ordering consumes only that - trait. -- Repair-armed sessions refuse this action without changing repair state. -- CLI help documents ADR 0017's `--record-as` workflow and warns that unparameterized fill/type inputs - remain literal script content. -- Provider-backed integration scenarios cover the public daemon route, and live iOS and Android runs - prove the saved artifact and post-save session behavior on real backends. +- An armed session publishes `open` plus target-annotated actions without `close`. + It returns the final path, remains active, and can continue to accept commands. +- The artifact replays from a cold start, completes its destination guard, returns the live session id, and accepts a subsequent command on that session. +- Every action in ADR 0012's existing target-binding command set has canonical identity evidence, and no unresolved `@ref` reaches disk. + Since #1349, the destination guard additionally requires verified recorded landmark identity. + A reshuffled-screen false-pass regression proves that replay fails closed. +- Existing-target refusal preserves the original bytes. + `--force` replaces atomically. + A failed publish remains retryable. +- After PUBLISHED, later ordinary actions remain usable, and a repeated `session save-script` fails. + Plain `open --relaunch` cannot re-arm. + `open --save-script` is rejected before app dispatch. +- In ABORTED/PUBLISHED, `close --save-script[=]` is rejected before platform close. + Plain `close` tears down without writing. + The close of an unpublished ARMED recording preserves the current close-time publication behavior. +- On a never-armed session (2026-08-02 amendment), `close --save-script` is likewise rejected before platform close or filesystem work. + The refusal names `open --save-script` as the recovery. + Plain `close` still tears down without writing. + The rejected request does not delete the session. +- Descriptor completeness tests classify the mutation effect of every recordable request, including request-sensitive read-only/mutating subcommands. + Destination-guard ordering consumes only that trait. +- Repair-armed sessions refuse this action without a change to repair state. +- CLI help documents ADR 0017's `--record-as` workflow. + CLI help warns that unparameterized fill/type inputs remain literal script content. +- Provider-backed integration scenarios cover the public daemon route. + Live iOS and Android runs prove the saved artifact and the post-save session behavior on real backends. diff --git a/docs/adr/0017-parameterized-recorded-inputs.md b/docs/adr/0017-parameterized-recorded-inputs.md index 3daf3d3afd..a5ee3e419d 100644 --- a/docs/adr/0017-parameterized-recorded-inputs.md +++ b/docs/adr/0017-parameterized-recorded-inputs.md @@ -4,22 +4,44 @@ Accepted +## Rules at a glance + +- The fill-only `recordAs` input parameterizes a recorded fill value. + The CLI spells it `--record-as `. +- The live fill dispatch receives the literal. + The public response, recording state, session event, and published script of the fill use `${VAR}`. +- Variable names use replay's uppercase `[A-Z_][A-Z0-9_]*` grammar. + `AD_*` remains reserved for runtime built-ins. +- `--record-as` is rejected unless script recording is armed. + `--record-as` cannot be combined with `--no-record`. +- Sensitivity is never inferred from a selector, label, field name, or value shape. + The caller names each mapping explicitly. +- If replay dispatches an authored fill whose complete text token is a `${VAR}` placeholder, replay derives `recordAs=VAR`. + Thus, a replay that is recorded or repaired retains the original placeholder. + Embedded interpolation remains ordinary script input. +- For the lifetime of one recording session, no later recorded action's own result payload or `target-v1`/`targets-v1` identity evidence may re-serialize an app-rendered echo of an already-parameterized literal (#1398). + The explicit scope limits of the #1398 amendment apply. +- Unparameterized fill/type values remain literal script content. + An author's own explicitly typed selector or positional text is not scanned. + ## Context -Ordinary `.ad` recording persists executable inputs. A literal `fill` value therefore becomes part of -the published script, including any password or token supplied during authoring. Diagnostic and event -redaction cannot make that executable artifact safe because replay still needs the value. +Ordinary `.ad` recording persists executable inputs. +A literal `fill` value therefore becomes part of the published script. +This includes any password or token supplied during authoring. +Diagnostic and event redaction cannot make that executable artifact safe, because replay still needs the value. Native replay already resolves `${VAR}` from `--env`/`-e` and `AD_VAR_*` immediately before dispatch. -The missing boundary is authoring: the live app needs the literal, while recording state and every -publication write must receive only the placeholder. Issue -[#1348](https://github.com/callstack/agent-device/issues/1348) defines this requirement; ADR 0016's -active-session publication deliberately excluded secret-bearing journeys until it was solved. +The missing boundary is authoring. +The live app needs the literal, while recording state and every publication write must receive only the placeholder. +Issue [#1348](https://github.com/callstack/agent-device/issues/1348) defines this requirement. +ADR 0016's active-session publication deliberately excluded secret-bearing journeys until #1348 was solved. ## Decision -Add a fill-only `recordAs` input, spelled `--record-as ` on the CLI and exposed as `recordAs` by -the Node and MCP fill contracts: +Add a fill-only `recordAs` input. +The CLI spells it `--record-as `. +The Node and MCP fill contracts expose it as `recordAs`: ```sh export AD_VAR_PASSWORD='' @@ -29,156 +51,167 @@ agent-device wait 'role="heading" label="Home"' agent-device session save-script ``` -The live fill dispatch receives the literal. Its public response, recording state, session event, and -published script use `${PASSWORD}`. Replay later resolves the placeholder from `AD_VAR_PASSWORD` or -`--env PASSWORD=` before the fill is dispatched. +The live fill dispatch receives the literal. +Its public response, recording state, session event, and published script use `${PASSWORD}`. +Later, replay resolves the placeholder from `AD_VAR_PASSWORD` or `--env PASSWORD=` before the fill is dispatched. -Variable names use replay's uppercase `[A-Z_][A-Z0-9_]*` grammar. `AD_*` remains reserved for runtime -built-ins. `--record-as` is rejected unless script recording is armed, and it cannot be combined with -`--no-record`. Ordinary fills without the option keep literal recording behavior. Sensitivity is never -inferred from a selector, label, field name, or value shape. +Variable names use replay's uppercase `[A-Z_][A-Z0-9_]*` grammar. +`AD_*` remains reserved for runtime built-ins. +`--record-as` is rejected unless script recording is armed. +`--record-as` cannot be combined with `--no-record`. +Ordinary fills without the option keep literal recording behavior. +Sensitivity is never inferred from a selector, label, field name, or value shape. ### Mapping contract -The caller names the mapping explicitly. Reusing `PASSWORD` deterministically emits `${PASSWORD}`; -using `API_TOKEN` emits `${API_TOKEN}`. There is no secret-to-name table and no literal comparison or -deduplication state. This both avoids retaining a second copy of the secret and makes distinct-value -naming an authoring decision visible in the script. +The caller names the mapping explicitly. +Reuse of `PASSWORD` deterministically emits `${PASSWORD}`. +Use of `API_TOKEN` emits `${API_TOKEN}`. +There is no secret-to-name table and no literal comparison or deduplication state. +This rule avoids the retention of a second copy of the secret. +It also makes distinct-value naming an authoring decision that is visible in the script. ### Data-flow boundary -The literal exists only on the live request path long enough to execute the interaction: +The literal exists only on the live request path, and only long enough to execute the interaction: -1. The request scope registers it as an explicit diagnostics secret before platform work. This is in - addition to key-name redaction, because an opaque value need not look secret. +1. Before platform work, the request scope registers the literal as an explicit diagnostics secret. + This is in addition to key-name redaction, because an opaque value need not look secret. 2. The platform interaction receives the literal. -3. Before response/event/recording retention, the fill result's semantic `text` field becomes - `${VAR}` and every occurrence in a post-fill `refLabel` is parameterized. Stable selector/ref - provenance is preserved byte-for-byte. - Untrusted backend extras and nested settle output parameterize every occurrence of the supplied - literal in both keys and values; post-fill selector-chain candidates are dropped only when a - parsed `text`, `label`, or `value` term semantically contains it. Known response and settle - schema fields are the explicit structural boundary and retain their names and provenance values. - Because inline whitespace replacement cannot distinguish a sensitive value from ordinary - separators, an untrusted string or key containing a whitespace-only literal collapses to the - placeholder instead. -4. At `recordActionEntry`, the recorder reconstructs the fill's literal positional, writes only the - placeholder into `SessionAction`, and parameterizes the semantic result field plus arbitrary - backend/settle echoes again. ADR 0012 `target-v1` evidence keeps its structure; exact - value-bearing accessibility labels become the placeholder without rewriting identity fragments - that merely contain the same characters. The `recordAs` control flag itself is not serialized - into the script. Parameterization is idempotent: this second pass preserves placeholders inserted - at the response boundary even when the literal is part of the placeholder's variable name or is - `$`. -5. The existing ADR 0012/0016 writer receives an already-parameterized action. Its selector provenance, - portability checks, same-directory temp write, and atomic publication algorithm are unchanged. - -Because the target and temp file are written from the same prepared script string, neither can contain -the literal once this pre-publication boundary has run. +3. Before response/event/recording retention, the semantic `text` field of the fill result becomes `${VAR}`. + Every occurrence in a post-fill `refLabel` is parameterized. + Stable selector/ref provenance is preserved byte-for-byte. + Untrusted backend extras and nested settle output parameterize every occurrence of the supplied literal in both keys and values. + Post-fill selector-chain candidates are dropped only when a parsed `text`, `label`, or `value` term semantically contains the literal. + Known response and settle schema fields are the explicit structural boundary, and they retain their names and provenance values. + Inline whitespace replacement cannot distinguish a sensitive value from ordinary separators. + Thus, an untrusted string or key that contains a whitespace-only literal collapses to the placeholder instead. +4. At `recordActionEntry`, the recorder reconstructs the literal positional of the fill and writes only the placeholder into `SessionAction`. + The recorder parameterizes the semantic result field plus arbitrary backend/settle echoes again. + ADR 0012 `target-v1` evidence keeps its structure. + Exact value-bearing accessibility labels become the placeholder. + Identity fragments that merely contain the same characters are not rewritten. + The `recordAs` control flag itself is not serialized into the script. + Parameterization is idempotent. + This second pass preserves placeholders inserted at the response boundary, even when the literal is part of the variable name of the placeholder or is `$`. +5. The existing ADR 0012/0016 writer receives an already-parameterized action. + Its selector provenance, portability checks, same-directory temp write, and atomic publication algorithm are unchanged. + +The target and the temp file are written from the same prepared script string. +Thus, neither can contain the literal once this pre-publication boundary has run. ### Replay and repair -Missing variables retain replay's existing fail-loud behavior: `${VAR}` resolution throws before the -corresponding action is invoked. When replay dispatches an authored fill whose complete text token is a -`${VAR}` placeholder, its daemon-only provenance channel derives `recordAs=VAR` for that live request. -This derivation uses the unresolved source action, independently of whether the resolved value is empty -or whitespace. If the replay is itself being recorded or repaired, the recorder therefore retains the -original placeholder rather than serializing the expanded value. Embedded interpolation remains -ordinary script input; safe authoring emits one complete placeholder token. +Missing variables retain replay's existing fail-loud behavior. +`${VAR}` resolution throws before the corresponding action is invoked. +If replay dispatches an authored fill whose complete text token is a `${VAR}` placeholder, replay's daemon-only provenance channel derives `recordAs=VAR` for that live request. +This derivation uses the unresolved source action, independently of whether the resolved value is empty or whitespace. +Thus, if the replay is itself being recorded or repaired, the recorder retains the original placeholder instead of serializing the expanded value. +Embedded interpolation remains ordinary script input. +Safe authoring emits one complete placeholder token. ### Session-scoped echo protection (amendment, #1398) -> **Status: accepted.** Amends the data-flow boundary above from fill-step-scoped to -> recording-session-scoped, per issue [#1398](https://github.com/callstack/agent-device/issues/1398). - -The boundary above protects only the originating fill's own request/recording path. After -[#1349](https://github.com/callstack/agent-device/issues/1349), a later read-only action — -`wait`'s landmark-mode evidence, `is`, `get` — records `target-v1` identity evidence and a result -payload independently, computed from whatever the app renders at that later step. That capture has no -memory of an earlier fill's literal, so an app-rendered echo of it (the filled field's own displayed -value; a search result, validation message, or confirmation label containing it; a destination landmark -whose accessible label includes it) can re-enter session state and publication through a completely -different, unparameterized action — even though the originating fill was parameterized. - -**The guarantee is now recording-session-scoped, not fill-step-scoped.** For the lifetime of one -recording session, no later recorded action's own result payload or `target-v1`/`targets-v1` identity -evidence may re-serialize an app-rendered echo of a literal the session already parameterized. - -The mechanism is the smallest explicit, ephemeral state that can recognize a later echo: -`SessionState.recordedFillLiterals`, an in-memory `Map` populated only by the SAME -(literal, placeholder) pair a `fill --record-as` entry already computes for its own boundary above — one -entry per parameterized fill, added only after that fill's own entry has been recorded. It is never -serialized (not to the script, the session event log, or diagnostics), has no read API beyond the -recorder that owns it, and disappears with the session. - -This is not a reversal of the "no secret-to-name table" rule in the Mapping contract above. That rule -rejects retaining state to influence a *naming* decision — comparing a new `--record-as` value against -prior ones to infer or deduplicate a variable name. `recordedFillLiterals` never informs a naming -decision: every entry's placeholder is exactly the name the author already chose, and the map is -consulted only to recognize that same, already-named value reappearing in unrelated later evidence — a -redaction lookup, not a secret registry with a naming or comparison API. +> **Status: accepted.** This amendment changes the data-flow boundary above from fill-step-scoped to recording-session-scoped, per issue [#1398](https://github.com/callstack/agent-device/issues/1398). + +The boundary above protects only the request/recording path of the originating fill itself. +After [#1349](https://github.com/callstack/agent-device/issues/1349), a later read-only action records `target-v1` identity evidence and a result payload independently. +These read-only actions are `wait` with landmark-mode evidence, `is`, and `get`. +That capture comes from whatever the app renders at that later step. +The capture has no memory of the literal of an earlier fill. +Thus, an app-rendered echo of the literal can re-enter session state and publication through a completely different, unparameterized action. +This can occur even though the originating fill was parameterized. +Examples of such an echo: + +- The displayed value of the filled field itself. +- A search result, validation message, or confirmation label that contains the literal. +- A destination landmark whose accessible label includes the literal. + +**The guarantee is now recording-session-scoped, not fill-step-scoped.** +For the lifetime of one recording session, no later recorded action's own result payload or `target-v1`/`targets-v1` identity evidence may re-serialize an app-rendered echo of a literal that the session already parameterized. + +The mechanism is the smallest explicit, ephemeral state that can recognize a later echo. +This state is `SessionState.recordedFillLiterals`, an in-memory `Map`. +Only the SAME (literal, placeholder) pair that a `fill --record-as` entry already computes for its own boundary above populates the map. +The map gets one entry for each parameterized fill, added only after the entry of that fill itself has been recorded. +The map is never serialized: not to the script, the session event log, or diagnostics. +The map has no read API beyond the recorder that owns it. +The map disappears with the session. + +This is not a reversal of the "no secret-to-name table" rule in the Mapping contract above. +That rule rejects the retention of state to influence a *naming* decision. +An example is a comparison of a new `--record-as` value with prior values to infer or deduplicate a variable name. +`recordedFillLiterals` never informs a naming decision. +The placeholder of every entry is exactly the name that the author already chose. +The map is consulted only to recognize that same, already-named value when it reappears in unrelated later evidence. +It is a redaction lookup, not a secret registry with a naming or comparison API. **Two treatments, by data class:** -- **Result/event/backend-output fields** (display data, never compared at replay): content-aware - substring redaction, reusing the same recursive backend-output scrub the fill boundary already applies - to its own entry, generalized over every literal registered so far in the session. The substitution - itself is one placeholder-safe left-to-right pass over each string, not N sequential full-string passes - — a naive sequential pass over the same value can corrupt an EARLIER pair's just-inserted placeholder - when a LATER pair's literal happens to be a substring of it (register `somethinglong -> ${ABC}`, then - `ABC -> ${OTHER}`, and a second pass over a value already rewritten to `${ABC}` matches "ABC" *inside* - that token, producing `${${OTHER}}`). The single pass tries every registered literal longest-first at - each position — so one registered value that is a substring of another (a username that is a prefix of - a password) is never partially consumed by the shorter pair — and never re-scans text it has already - emitted, so no literal can ever be matched inside another pair's placeholder token in either direction. -- **`target-v1`/`targets-v1` identity evidence** (`label`, `ancestry[].label`, `scrollRegion.label` — the - fields replay's own classification compares): never silently text-substituted while still claiming a - trustworthy identity. Replay compares recorded identity against the *live* tree, which legitimately - re-renders the real value again at replay time; a placeholder written into a recorded identity field - could therefore never verify correctly. Instead: - - **Landmark mode (`wait`).** An echo is treated exactly like #1349's existing identity-empty case: no - annotation is recorded, and the wait keeps its selector-existence semantics. Because ADR 0016's - destination guard requires `verification: "verified"`, a landmark whose only identity is a - parameterized-value echo simply stops qualifying as a guard — `session save-script` refuses it with - the existing "record a selector-targeted wait on a labeled or id-bearing landmark" recovery, - reproducing the motivating scenario's real resolution (switching the guard to the stable `Apps` - landmark) as an enforced outcome rather than an authoring convention. - - **Action mode (`get`, `is`, mutating element-targeting actions).** ADR 0012/0016 require identity - evidence for every element-targeting recorded action and forbid silently dropping it, so an echo here - is never dropped. The literal-bearing label(s) are redacted to the placeholder (content-aware, - substring-based — unlike the exact-match fill-boundary redaction, because a cross-step echo is - typically surrounded by app-authored text such as "Welcome, ``") and `verification` is - downgraded to `"unverifiable"` — the same fail-closed downgrade decision 3's writer-parser invariant - already uses for an oversized payload. That fails the action's replay loudly - (`identity-unverifiable`) rather than silently weakening it to selector-only matching or publishing a - label that could never match again. +- **Result/event/backend-output fields** (display data, never compared at replay). + These fields get content-aware substring redaction. + This redaction reuses the same recursive backend-output scrub that the fill boundary already applies to its own entry. + The redaction is generalized over every literal registered so far in the session. + The substitution itself is one placeholder-safe left-to-right pass over each string, not N sequential full-string passes. + A naive sequential pass over the same value can corrupt the just-inserted placeholder of an EARLIER pair. + This occurs when the literal of a LATER pair happens to be a substring of that placeholder. + For example, register `somethinglong -> ${ABC}`, then `ABC -> ${OTHER}`. + A second pass over a value already rewritten to `${ABC}` matches "ABC" *inside* that token, and produces `${${OTHER}}`. + The single pass tries every registered literal longest-first at each position. + Thus, one registered value that is a substring of another (a username that is a prefix of a password) is never partially consumed by the shorter pair. + The single pass also never re-scans text that it has already emitted. + Thus, no literal can ever be matched inside the placeholder token of another pair, in either direction. +- **`target-v1`/`targets-v1` identity evidence** (`label`, `ancestry[].label`, `scrollRegion.label`: the fields that replay's own classification compares). + This evidence is never silently text-substituted while it still claims a trustworthy identity. + Replay compares recorded identity against the *live* tree. + The live tree legitimately re-renders the real value again at replay time. + Thus, a placeholder written into a recorded identity field could never verify correctly. + Instead: + - **Landmark mode (`wait`).** An echo is treated exactly like the existing identity-empty case of #1349. + No annotation is recorded, and the wait keeps its selector-existence semantics. + ADR 0016's destination guard requires `verification: "verified"`. + Thus, a landmark whose only identity is an echo of a parameterized value simply stops qualifying as a guard. + `session save-script` refuses it with the existing "record a selector-targeted wait on a labeled or id-bearing landmark" recovery. + This reproduces the real resolution of the motivating scenario (a change of the guard to the stable `Apps` landmark). + It makes that resolution an enforced outcome, not an authoring convention. + - **Action mode (`get`, `is`, mutating element-targeting actions).** ADR 0012/0016 require identity evidence for every element-targeting recorded action and forbid silently dropping it. + Thus, an echo here is never dropped. + The literal-bearing label(s) are redacted to the placeholder. + This redaction is content-aware and substring-based, unlike the exact-match fill-boundary redaction. + The reason is that app-authored text typically surrounds a cross-step echo, such as "Welcome, ``". + `verification` is also downgraded to `"unverifiable"`. + This is the same fail-closed downgrade that the writer-parser invariant of decision 3 already uses for an oversized payload. + That downgrade makes the replay of the action fail loudly (`identity-unverifiable`). + Replay does not silently weaken the action to selector-only matching, and the script does not publish a label that could never match again. **Explicit scope limits:** -- A whitespace-only or empty resolved fill value is excluded from the session-wide registry; it keeps - only the existing fill-step-scoped protection above. Collapsing arbitrary later strings on a value with - no discriminating content is a disproportionate readability cost for a value that reveals nothing - distinctive if echoed, and would corrupt unrelated short incidental substrings throughout the rest of - the recording. -- The originating fill's own recorded entry is protected only by the existing exact-match fill-boundary - pass; it is never passed through the coarser, substring-based session-wide pass using the pair it just - contributed. The session-wide guarantee is for later, distinct actions, exactly as #1398 frames it. -- An author's own explicitly typed selector or positional text is not scanned — only derived, resolved - evidence and result payloads are. An author who types the secret directly into a new selector is making - the same choice as an unparameterized fill/type value; that remains literal script content. -- The registry is keyed by literal, so two distinct `--record-as` names that happen to share the same - typed value (a password/confirm-password pair, say) are genuinely indistinguishable from a later - echo's perspective — the literal alone cannot say which fill produced it. The first-registered name - wins deterministically; the literal is redacted either way, so this only affects which placeholder - name a later echo is attributed to, never whether the value is protected. +- A whitespace-only or empty resolved fill value is excluded from the session-wide registry. + It keeps only the existing fill-step-scoped protection above. + The collapse of arbitrary later strings on a value with no discriminating content has a disproportionate readability cost. + Such a value reveals nothing distinctive if echoed. + The collapse would also corrupt unrelated short incidental substrings throughout the rest of the recording. +- The originating fill's own recorded entry is protected only by the existing exact-match fill-boundary pass. + That entry never goes through the coarser, substring-based session-wide pass with the pair that it just contributed. + The session-wide guarantee is for later, distinct actions, exactly as #1398 frames it. +- An author's own explicitly typed selector or positional text is not scanned. + Only derived, resolved evidence and result payloads are scanned. + An author who types the secret directly into a new selector makes the same choice as an unparameterized fill/type value. + That text remains literal script content. +- The registry is keyed by literal. + Two distinct `--record-as` names can share the same typed value (for example, a password/confirm-password pair). + From the perspective of a later echo, these names are genuinely indistinguishable, because the literal alone cannot tell which fill produced it. + The first-registered name wins deterministically. + The literal is redacted either way. + Thus, this affects only the placeholder name to which a later echo is attributed, never whether the value is protected. ## Consequences - Secret-bearing login/bootstrap scripts can use ADR 0016 active publication safely. -- The caller must opt in for every sensitive fill; unparameterized fill/type values remain literal - script content and help warns accordingly. -- `type` and mutating `find ... fill|type` parameterization are not added by this decision. They require - their own surface and provenance design if needed. -- No new publication format, variable store, or sensitivity heuristic is introduced. The #1398 amendment - adds one ephemeral, non-serialized, per-session redaction map with no naming or lookup-by-name API — - never a persistent secret registry. +- The caller must opt in for every sensitive fill. + Unparameterized fill/type values remain literal script content, and help warns accordingly. +- This decision does not add `type` and mutating `find ... fill|type` parameterization. + If needed, they require their own surface and provenance design. +- No new publication format, variable store, or sensitivity heuristic is introduced. + The #1398 amendment adds one ephemeral, non-serialized, per-session redaction map with no naming or lookup-by-name API. + This map is never a persistent secret registry. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1c56e3cb49..f0384d98e0 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -9,7 +9,7 @@ | [0005 iOS Runner Interaction Lifecycle](0005-ios-runner-interaction-lifecycle.md) | XCTest runner sessions, leases, adoption, idle-stop | | [0006 Daemon RPC Protocol Version](0006-daemon-rpc-protocol-version.md) | remote daemon HTTP/JSON-RPC compatibility | | [0007 Remote Device Leases](0007-remote-device-leases.md) | leases, tenancy, provider-owned devices | -| [0008 Command Descriptor Registry](0008-command-descriptor-registry.md) | adding/changing a command, any surface projection (CLI/MCP/client/batch), timeout policy | +| [0008 Command Descriptor Registry](0008-command-descriptor-registry.md) | command additions or changes, any surface projection (CLI/MCP/client/batch), timeout policy | | [0009 Apple Platform Consolidation](0009-apple-platform-consolidation.md) | Apple platform family, apple/appleOs axes, the apple-leak guard | | [0010 Error system conventions](0010-error-system.md) | error codes, hints, normalizeError, typed error signals | | [0011 Interaction Guarantee Contract](0011-interaction-guarantee-contract.md) | interaction dispatch paths, fast paths, guards, the guarantee matrix, parity tables | @@ -17,37 +17,39 @@ | [0013 Unified Gesture Plans](0013-unified-gesture-plans.md) | gesture API/routing, contact topology, multi-touch geometry, native pointer injection, two-finger pan | | [0014 Session Ref-Frame Lifetime](0014-session-ref-frame-lifetime.md) | ref authorization epochs, complete/partial issuance, pre-side-effect expiration, replay/batch compatibility, and cross-platform stale-mutation policy | | [0015 Direct Maestro Compatibility Engine](0015-direct-maestro-engine.md) | Maestro YAML parsing/execution, compatibility observation policy, conformance, performance gates, gesture integration | -| [0016 Active-Session Script Publication](0016-active-session-script-publication.md) | publishing an armed open-to-destination `.ad` script without closing its live session | -| [0017 Parameterized Recorded Inputs](0017-parameterized-recorded-inputs.md) | safely authoring sensitive fill inputs as `${VAR}` placeholders across recording, replay, and repair | +| [0016 Active-Session Script Publication](0016-active-session-script-publication.md) | publication of an armed open-to-destination `.ad` script that retains its live session | +| [0017 Parameterized Recorded Inputs](0017-parameterized-recorded-inputs.md) | safe authoring of sensitive fill inputs as `${VAR}` placeholders across recording, replay, and repair | | [0018 Unified Request Event Journal (Proposed)](0018-unified-event-journal.md) | event/diagnostic vocabulary, journal scopes and sinks, progress-channel separation, observability-only state | | [0019 Request-Bound Platform Runtime](0019-request-bound-platform-runtime.md) | platform-package boundaries/composition, device discovery, runtime facts/facets, request binding, provider ownership, platform-shaped session resources, durable reattachment, daemon-handler migration | | [0020 Composable Recorded Fragments (Proposed)](0020-composable-recorded-fragments.md) | lifecycle-free recorded fragment capture/composition, entry guards, fragment-local addresses/digests, staleness, and native `.ad`/Maestro composition | -| [0021 Simlock-Leased Devices Through Ordinary Local Runtimes](0021-host-simlock-managed-device-allocation.md) | driving a Simlock-leased simulator or emulator, why agent-device has no allocator integration, and the withdrawn in-process Host design | -| [0022 Daemon — Platform Runtime Coupling Audit and Ownership Ratchets](0022-daemon-platform-runtime-coupling.md) | daemon imports of root `platform-runtime-*` modules, the retired R76 edge classification now collapsed to a coupling ban, R75 session-authority ratchet, entry-to-platform hop routes and roles | -| [0023 End-State Entry-to-Platform Hop Trace](0023-end-state-hop-trace.md) | the hop counting definition, the ordered `press`/Android and dual-arm `snapshot`/iOS chains, per-hop roles, and the deletion test behind the numbers ADR 0019 and ADR 0022 quote | -| [0024 Screen Recording — One Coordinator, Two Facts (Proposed)](0024-screen-recording-provable-signal.md) | `record start`/`stop` lifecycle, the export-exists / recorder-stopped split, backend stop-and-collect contract, recovery through the coordinator, the ADR 0019 §5 amendment and the failed-finish cleanup fix | -| [0025 Foldable Apple Panels — Capture the Lit Panel](0025-foldable-apple-panels.md) | iPhone Duo outer/inner panels, CoreDevice display info as the only screen authority, why `simctl`'s implicit display default captures a black panel, derived (never asserted) pose, the `fold` pose control, and the Android emulator posture amendment | -| [0026 Scroll Clip Authority — Ownership Is the Parent Edge](0026-scroll-clip-authority.md) | iOS scroll indicators and the visible band, why ownership is read from the parent edge instead of an ancestor walk, what may remove a node from an interactive snapshot, and ejection dispositions | -| [0027 Descriptor Root Size vs Eager-Closure Budget](0027-descriptor-root-vs-eager-closure-budget.md) | accepted: the eager-closure budget tolerates a statement-neutral hub split only (containment + added-module novelty + flat top-level-statement weight) and nothing else; splitting `packages/command-registry/src/registry.ts` is unblocked | -| [0028 Capability-Family Cell Vocabulary — One Runtime Source (Proposed)](0028-capability-family-cell-vocabulary.md) | adding a capability operation family, `UnavailablePlatformRuntimeFacts` / `UNAVAILABLE_CELLS`, `INTERACTOR_OPERATIONS`, and why an eight-package fan-out recurs | -| [0029 Daemon Policy](0029-daemon-policy.md) | `AGENT_DEVICE_DAEMON_POLICY`, confining a daemon's commands, devices, or device shutdown, and where operator rules are enforced for batch/replay steps | +| [0021 Simlock-Leased Devices Through Ordinary Local Runtimes](0021-host-simlock-managed-device-allocation.md) | operation of a Simlock-leased simulator or emulator, the reason for no agent-device allocator integration, and the withdrawn in-process Host design | +| [0022 Daemon — Platform Runtime Coupling Audit and Ownership Ratchets](0022-daemon-platform-runtime-coupling.md) | daemon imports of root `platform-runtime-*` modules, the coupling ban that replaces R76 edge classification, the R75 session-authority ratchet, entry-to-platform hop routes and roles | +| [0023 End-State Entry-to-Platform Hop Trace](0023-end-state-hop-trace.md) | the definition of hop counts, ordered `press`/Android and dual-arm `snapshot`/iOS chains, per-hop roles, and the deletion test for numbers in ADR 0019 and ADR 0022 | +| [0024 Screen Recording — One Coordinator, Two Facts (Proposed)](0024-screen-recording-provable-signal.md) | `record start`/`stop` lifecycle, separate export-exists / recorder-stopped facts, backend stop-and-collect contract, coordinator recovery, the ADR 0019 §5 amendment, and the failed-finish cleanup fix | +| [0025 Foldable Apple Panels — Capture the Lit Panel](0025-foldable-apple-panels.md) | iPhone Duo outer/inner panels, CoreDevice display info as sole screen authority, black-panel capture from `simctl`'s implicit display default, derived (never asserted) pose, `fold` pose control, and the Android emulator posture amendment | +| [0026 Scroll Clip Authority — Ownership Is the Parent Edge](0026-scroll-clip-authority.md) | iOS scroll indicators and the visible band, parent-edge ownership instead of ancestor traversal, permitted node removal from interactive snapshots, and ejection dispositions | +| [0027 Descriptor Root Size vs Eager-Closure Budget](0027-descriptor-root-vs-eager-closure-budget.md) | accepted: the eager-closure budget permits only a statement-neutral hub split (containment + added-module novelty + flat top-level-statement weight); `packages/command-registry/src/registry.ts` may be split | +| [0028 Capability-Family Cell Vocabulary — One Runtime Source (Proposed)](0028-capability-family-cell-vocabulary.md) | capability operation family additions, `UnavailablePlatformRuntimeFacts` / `UNAVAILABLE_CELLS`, `INTERACTOR_OPERATIONS`, and the reason for repeated changes across eight packages | +| [0029 Daemon Policy](0029-daemon-policy.md) | `AGENT_DEVICE_DAEMON_POLICY`, limits on daemon commands, devices, or device shutdown, and where operator rules are enforced for batch/replay steps | | [0030 Process Lock Exclusion](0030-process-lock-exclusion.md) | process-lock publication/reclaim/release, retained mutation guards, and the single-protocol upgrade boundary | -| [0031 macOS Native App Backend](0031-macos-native-app-backend.md) | `AGENT_DEVICE_MACOS_APP_BACKEND`, driving macOS app sessions without XCTest Automation Mode, why pointer actions are accessibility actions only, and the ghost cursor | -| [0032 Layering Graph Engine](0032-layering-graph-engine.md) | evaluating replacement of `scripts/layering/` graph rules with dependency-cruiser or fallow `boundaries`, why graph-based layering checks share one replaceable import model, the measured edge-set and parity gaps, and the revisit triggers | -| [0033 Collocation decision table](0033-collocation-decision-table.md) | moving or adding a file listed in umbrella #3276 §5, re-proposing a move that table refused, reading a zone's collocation decisions | +| [0031 macOS Native App Backend](0031-macos-native-app-backend.md) | `AGENT_DEVICE_MACOS_APP_BACKEND`, macOS app sessions without XCTest Automation Mode, the accessibility-only pointer action rule, and the ghost cursor | +| [0032 Layering Graph Engine](0032-layering-graph-engine.md) | replacement evaluation for `scripts/layering/` graph rules with dependency-cruiser or fallow `boundaries`, the reason that graph-based layering checks share one replaceable import model, the measured edge-set and parity gaps, and the revisit triggers | +| [0033 Collocation decision table](0033-collocation-decision-table.md) | moves or additions for files in umbrella #3276 §5, renewed proposals for a move that the table refused, and a zone's collocation decisions | | [0034 Provider plugin registry](0034-provider-plugin-registry.md) | provider plugins, the `agent-device/plugins` contract, plugin sources and order, duplicate provider ids, plugin failure isolation, the daemon plugin inventory and `daemon status` | -ADRs record *why*; the registries and gates they describe are the living source of truth — when -prose and a registry disagree, the registry wins and the ADR needs a follow-up. +ADRs record rationale. +The registries and gates they describe are the current source of truth. +If prose and a registry disagree, the registry takes precedence. +The ADR then needs a follow-up. -Shape conventions, so consulting an ADR stays cheap: +Use these shape conventions to keep ADRs inexpensive to consult: -- **Normative rules first, terse.** Status, then a "Rules at a glance" summary a reader can stop - after; full contracts and rationale below it. -- **Rationale and refuted alternatives stay in the ADR** — they are what stops re-litigating - settled ideas — but below the fold. -- **Process history is deleted once complete, not archived in-file.** Migration plans, per-step - landing tables, and point-in-time status change-logs go to git history; the Status section keeps - one line saying so plus any accepted, still-relevant waiver or evidence gap. -- Once a rule is gate-enforced, the ADR keeps the why and points at the gate rather than restating - the rule's details. +- **Put normative rules first.** Put Status before a terse "Rules at a glance" summary. + The summary lets readers stop there. + Put full contracts and rationale below it. +- **Retain rationale and rejected alternatives.** They prevent repeated discussion of settled decisions. + Put them below the summary. +- **Process history is deleted once complete.** Keep completed migration plans, per-step implementation tables, and point-in-time status logs only in git history. + In Status, keep one line that states this deletion. + In Status, also keep any accepted waiver or evidence gap that is still relevant. +- If a gate enforces a rule, retain its rationale and a gate pointer instead of repeated rule details.