From 445ff0841ed2588d5ab00c3e3060d2d977699bd6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:52:44 +0200 Subject: [PATCH 01/55] docs(adr): write ADR 0001 in ASD-STE100 --- ...01-provider-first-integration-scenarios.md | 226 +++++++++++++++--- 1 file changed, 188 insertions(+), 38 deletions(-) diff --git a/docs/adr/0001-provider-first-integration-scenarios.md b/docs/adr/0001-provider-first-integration-scenarios.md index ab2f58666c..ba509f1b77 100644 --- a/docs/adr/0001-provider-first-integration-scenarios.md +++ b/docs/adr/0001-provider-first-integration-scenarios.md @@ -4,69 +4,219 @@ Accepted -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Provider-first scenario -> coverage and semantic provider operations remain accepted. The monolithic `Interactor` and the -> request callback-stack topology are superseded one command at a time by request-bound runtime -> facets and explicit provider ownership. +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Provider-first scenario coverage and semantic provider operations remain accepted. +> Request-bound runtime facets and explicit provider ownership supersede the monolithic `Interactor` and request callback-stack topology one command at a time. + +## Rules at a glance + +- For commands not yet migrated under ADR 0019, keep the legacy `Interactor` interface. +- For new command migrations, use bound runtime facets from ADR 0019. +- In provider-backed integration tests, run the real daemon request path. Replace only providers. +- By default, match provider transcripts without order. Use ordered transcripts only to test order. +- Keep HTTP coverage in a narrow suite for transport, auth, and response finalization. +- Keep synchronous host-tool calls local until their paths have semantic provider contracts. +- Mock-heavy handler tests should be deleted only after equivalent integration scenarios exist. ## Context -The test suite had many mocked daemon handler and dispatch unit tests. Those tests were expensive to maintain and skipped important behavior across request admission, locking, session state, handler routing, dispatch, Interactor resolution, and platform module command translation. +The test suite had many mocked unit tests for daemon handlers and dispatch. +These tests cost much to maintain and omitted important behavior in: + +- Request admission. +- Locking. +- Session state. +- Handler routing. +- Dispatch. +- Interactor resolution. +- Command translation in platform modules. -Android already had an ADB provider seam. Apple-family and Linux platform modules mostly called host tools or runner commands directly, which made device-free integration tests difficult without mocking too high in the stack. +Android already had an ADB provider boundary. +Apple-family and Linux platform modules mostly called host tools or runner commands directly. +Thus, device-free integration tests required mocks above important platform behavior. ## Decision -For commands not yet migrated under ADR 0019, keep `Interactor` as the legacy semantic interface -between dispatch and platform behavior. New command cutovers use ADR 0019's bound runtime facets. +For commands not yet migrated under ADR 0019, keep `Interactor` as the legacy semantic interface between dispatch and platform behavior. +For new command migrations, use bound runtime facets from ADR 0019. -For commands still on the legacy adapter, use these request-scoped provider seams below platform -modules: +For commands that still use the legacy adapter, use these request-scoped provider boundaries below platform modules: -- Android ADB provider -- Apple tool provider with semantic `simctl`, `devicectl`, macOS helper, and macOS host subproviders -- Apple runner provider -- Linux tool provider, with semantic desktop lifecycle, accessibility, clipboard, and screenshot subproviders -- App-log provider for semantic app log stream start/stop -- Recording provider for semantic local/remote screen-recording start operations that would otherwise be raw host processes +- Android ADB provider. +- Apple tool provider with semantic `simctl`, `devicectl`, macOS helper, and macOS host subproviders. +- Apple runner provider. +- Linux tool provider with semantic subproviders for desktop lifecycle, accessibility, clipboard, and screenshots. +- App-log provider for semantic start and stop of app log streams. +- Recording provider for semantic local and remote screen-recording starts that would otherwise use raw host processes. -Provider contracts should expose semantic operations when the platform intent is stable enough to name. Android already does this for install, pull, and port-reverse behavior. Apple tool execution has started moving in that direction with semantic `simctl`, `devicectl`, macOS helper, and macOS host runners; generic command execution remains as a local compatibility fallback for host-tool paths that have not been classified yet. The macOS host provider owns app opening, app inventory, clipboard, and appearance because remote desktop providers should not need to infer those intents from `open`, `find`, `plutil`, `pbcopy`, `pbpaste`, or `osascript`. Linux desktop lifecycle exposes semantic `openTarget` and `closeApp` operations because provider-backed integration covers those user workflows and remote desktop providers should not need to infer intent from `xdg-open`, binary launch, `wmctrl`, or `pkill`. Linux accessibility snapshots, clipboard, screenshots, and input synthesis are also semantic because the stable contracts are "capture this surface", "read/write clipboard text", "write a screenshot artifact", and "perform this user input", not the local implementation details `python3`/AT-SPI, `xclip`, `scrot`, `grim`, `xdotool`, or `ydotool`. App-log stream start/stop is semantic observability behavior rather than generic host process execution, so provider-backed integration injects an app-log provider instead of spawning `log stream` or `logcat`. iOS simulator screen recording now uses a recording provider because the stable contract is "start simulator recording for this device and output path", not the local implementation detail `xcrun simctl io recordVideo`. +When platform intent is sufficiently stable to name, provider contracts should expose semantic operations. +Android already does this for install, pull, and port-reverse behavior. +Apple tool execution has started this change through semantic `simctl`, `devicectl`, macOS helper, and macOS host runners. +For unclassified host-tool paths, generic command execution remains a local compatibility fallback. -Provider-backed integration tests run the real daemon request path and replace only those providers. Tests may use provider transcripts for platform command contracts and scenario transcripts for broader user workflows. Provider transcripts match calls as an unordered contract by default; use ordered transcripts only when ordering is the behavior under test. +The macOS host provider owns these operations: -Prefer an in-process provider-backed integration harness for broad scenarios: it should invoke the daemon request handler directly, preserving admission, locking, session state, handler routing, dispatch, platform modules, and provider seams without binding a TCP listener. Keep HTTP coverage as a narrow contract suite for JSON-RPC transport, auth, and response finalization. +- App open. +- App inventory. +- Clipboard access. +- Appearance control. -Request provider scoping is descriptor-driven inside the request router layer. A provider entry declares how it applies to the resolved device, how it installs its scoped provider, and what request-local data it exposes to handlers. Apple runner providers are request-id scoped when a request id exists; fallback by device alone is only allowed for unscoped local runner calls. +Remote desktop providers should not need to infer these intents from `open`, `find`, `plutil`, `pbcopy`, `pbpaste`, or `osascript`. +Linux desktop lifecycle exposes semantic `openTarget` and `closeApp` operations. +Provider-backed integration covers these user workflows. +Remote desktop providers should not need to infer intent from `xdg-open`, binary launch, `wmctrl`, or `pkill`. -Synchronous host-tool calls are intentionally not part of the provider seam. Any remaining sync Apple helper is local-only and must be converted before a remote/cloud provider can own that path. +Linux accessibility snapshots, clipboard access, screenshots, and input synthesis are also semantic. +Their stable contracts are: -Remaining generic Apple host-tool calls are intentionally local-only unless a new adapter creates pressure to promote them: perf process discovery/sampling (`mdfind`, `ps`), launch diagnostics (`plutil`, `otool`), local runner product build/signing (`swift`, `codesign`, sync `plutil`), local simulator UI launch (`open -a Simulator`), plist compatibility fallbacks, and fallback implementations behind semantic macOS providers. Provider-backed integration scenarios should avoid scripting those host commands for user workflows; a user-facing workflow that depends on one of them is naming pressure for a semantic provider method. +- Capture this surface. +- Read or write clipboard text. +- Write a screenshot artifact. +- Perform this user input. -## Alternatives Considered +These contracts describe intent independently of `python3`/AT-SPI, `xclip`, `scrot`, `grim`, `xdotool`, or `ydotool`. +App-log stream start and stop are semantic observability behavior. +Thus, provider-backed integration injects an app-log provider instead of starting `log stream` or `logcat`. +iOS simulator screen recording uses a recording provider. +Its stable contract starts simulator recording for a specified device and output path. +The local implementation uses `xcrun simctl io recordVideo`. -- Mock handlers or `dispatchCommand`: cheaper to write, but it skips request admission, locking, session state, and platform command translation, which were the main sources of test blind spots. -- Put the seam at `Interactor`: simpler and more uniform, but it bypasses platform modules and would not catch the iOS/Linux host-tool wiring issues that motivated this change. -- Start with a full semantic provider per platform operation: cleaner end state, but too much surface to name correctly in one pass. The migration starts where contracts already exist or where tests create pressure. -- Run every provider-backed integration scenario through HTTP: maximum end-to-end coverage, but it makes most scenarios pay for TCP setup, sandbox permissions, and transport timeouts even when transport behavior is not under test. +Provider-backed integration tests run the real daemon request path and replace only these providers. +Tests may use provider transcripts for platform command contracts. +Tests may use scenario transcripts for broader user workflows. +By default, provider transcripts match calls without order. +Use ordered transcripts only when order is the behavior under test. -## Consequences +Prefer an in-process harness with providers for broad integration scenarios. +The harness should invoke the daemon request handler directly without a TCP listener. +It should preserve: -Platform command translation remains covered by integration tests without requiring real devices. +- Admission. +- Locking. +- Session state. +- Handler routing. +- Dispatch. +- Platform modules. +- Provider boundaries. -The request router owns a provider registry seam, but platform-specific provider applicability remains localized in that registry. The registry composes provider scopes linearly so adding a platform does not require another nested wrapper chain. +Keep HTTP coverage as a narrow contract suite for JSON-RPC transport, auth, and response finalization. -New remote or cloud-backed adapters can implement neutral provider contracts without changing daemon, dispatch, or session contracts. Generic tool-provider fallbacks are an interim compatibility layer, not the target contract for cloud adapters. When a provider-backed integration scenario still scripts raw host commands, that is a signal to reassess whether the platform intent has become stable enough for a semantic provider method. +Descriptors control request provider scope inside the request router layer. +Each provider entry declares: -Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenario coverage exists. Unit tests remain appropriate for pure logic, parser matrices, selector matching, capability maps, and edge/error cases that integration tests would express poorly. +- How the provider applies to the resolved device. +- How the provider installs its scoped provider. +- Which request-local data the provider exposes to handlers. -The trade-off is coarser failure localization: a provider-backed integration scenario catches more of the real request path but may require more diagnosis than a narrow unit test. Scenario names and provider transcript entries should stay rooted in user workflows and real e2e examples so failures remain actionable. +If a request id exists, Apple runner providers use request-id scope. +Only unscoped local runner calls may use a fallback based on the device alone. -Coverage is expected to improve over the old handler-heavy unit suite, but the first migration does not meet the original 90% target. The current coverage denominator also excludes some entrypoint and configuration files, so coverage should be treated as a trend signal rather than proof that every public surface is exercised. +Synchronous host-tool calls are intentionally outside the provider boundary. +Each remaining synchronous Apple helper is local-only. +Before a remote or cloud provider can own that path, convert the helper. -Operational metrics are generated by `pnpm test:integration:progress`. CI runs `pnpm test:integration:progress:check` after integration tests so public-command coverage, device-observable workflow flag coverage, and public-flag classification cannot silently regress. The script is the source of truth for provider-backed integration size, handler-unit size, mock-heavy handler pressure, public-command coverage, command-family ownership, device-observable workflow flag coverage, provider transcript pressure, and low-coverage files after a coverage run. Config, remote transport, Metro preparation, parser/client-only, report-writing, and boot-fallback flags stay in their owning unit/CLI suites and are reported as explicit exclusions rather than silently missing from the denominator. Provider pressure separates semantic Apple `simctl`/`devicectl`, macOS helper, and macOS host usage from generic Apple host-tool usage, and separates semantic Linux desktop/accessibility/clipboard/screenshot/input usage from generic Linux tool usage, so remote-adapter pressure is visible without treating named subproviders as raw shell intent. +Remaining generic Apple host-tool calls intentionally stay local-only unless a new adapter requires semantic operations: -The progress CLI should stay a thin report and check runner over a progress model. The model owns discovery, coverage classification, provider-pressure accounting, and check-failure derivation; the CLI owns Markdown output and process exit behavior. The progress script runs as a Node type-stripped TypeScript script and should consume command metadata directly when the metadata API exposes the needed facet. Source parsing inside the model is an implementation detail, not the desired long-term interface, and should remain limited to facets that are not yet represented as runtime metadata, such as mapping typed client method calls back to command names. +- Perf process discovery and samples: `mdfind`, `ps`. +- Launch diagnostics: `plutil`, `otool`. +- Local runner product build and signature: `swift`, `codesign`, synchronous `plutil`. +- Local simulator UI launch: `open -a Simulator`. +- Plist compatibility fallbacks. +- Fallback implementations behind semantic macOS providers. -Every public command should have at least one provider-backed integration scenario that runs through the request router and request-scoped provider seams. Unit tests remain for parser matrices, selector matching, capability maps, malformed inputs, state machines, cleanup behavior, provider scope routing, and platform error boundaries. +Provider-backed integration scenarios should avoid scripts of these host commands for user workflows. +If a user-facing workflow depends on one, consider a semantic provider method for that intent. + +## Alternatives Considered + +- **Mock handlers or `dispatchCommand`.** These tests cost less to write. + They omit request admission, locking, session state, and platform command translation, where the main test gaps occurred. +- **Put the boundary at `Interactor`.** This is simpler and more uniform. + It bypasses platform modules and would miss the iOS/Linux host-tool connections that motivated this change. +- **Start with a full semantic provider per platform operation.** This gives a cleaner final design. + Too many operations require names in one change. + The migration starts where contracts exist or tests show a need. +- **Run every provider-backed integration scenario through HTTP.** This gives maximum end-to-end coverage. + Most scenarios would require TCP setup, sandbox permissions, and transport timeouts even when they do not test transport behavior. + +## Consequences -The temporary migration roadmap is complete and intentionally removed from the repository. Future work should be justified from live pressure in the progress script, failing tests, or concrete adapter needs rather than from a standing refactor backlog. +Integration tests continue to cover platform command translation without real devices. + +The request router owns a provider registry boundary. +The registry keeps platform-specific provider applicability local. +It composes provider scopes linearly. +Thus, a new platform does not require another chain of nested wrappers. + +New remote or cloud adapters can implement neutral provider contracts without changes to daemon, dispatch, or session contracts. +Generic tool-provider fallbacks provide interim compatibility. +They are not the target contract for cloud adapters. +If an integration scenario still scripts raw host commands, reassess whether the platform intent is sufficiently stable for a semantic provider method. + +Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenarios exist. +Unit tests remain appropriate for pure logic, parser matrices, selector matching, capability maps, and edge/error cases that integration tests express poorly. + +A provider-backed integration scenario covers more of the real request path. +However, its failures may require more diagnosis than failures in narrow unit tests. +Scenario names and provider transcript entries should describe user workflows and real e2e examples, so failures identify useful actions. + +Coverage is expected to improve over the old suite of handler unit tests. +However, the first migration does not meet the original 90% target. +The current coverage denominator also excludes some entrypoint and configuration files. +Thus, coverage should indicate trends instead of proving that tests exercise every public surface. + +`pnpm test:integration:progress` generates operational metrics. +After integration tests, CI runs `pnpm test:integration:progress:check` to prevent silent regressions in: + +- Public-command coverage. +- Device-observable workflow flag coverage. +- Public-flag classification. + +After a coverage run, the script is the source of truth for: + +- Provider-backed integration size. +- Handler-unit size. +- Mock-heavy handler pressure. +- Public-command coverage. +- Command-family ownership. +- Device-observable workflow flag coverage. +- Provider transcript pressure. +- Files with low coverage. + +These flag categories stay in their owning unit/CLI suites: + +- Config. +- Remote transport. +- Metro preparation. +- Parser/client-only. +- Report-writing. +- Boot-fallback. + +The report lists them as explicit exclusions from the denominator. +Provider pressure distinguishes semantic Apple `simctl`/`devicectl`, macOS helper, and macOS host usage from generic Apple host-tool usage. +It also distinguishes semantic Linux desktop/accessibility/clipboard/screenshot/input usage from generic Linux tool usage. +Thus, the report shows remote-adapter pressure without classifying named subproviders as raw shell intent. + +The progress CLI should remain a small report and check runner over a progress model. +The model owns discovery, coverage classification, provider-pressure accounting, and check-failure derivation. +The CLI owns Markdown output and process exit behavior. +The progress script runs as a Node type-stripped TypeScript script. +When the metadata API exposes the required facet, the script should consume command metadata directly. +Source parsing inside the model is an implementation detail, not the desired long-term interface. +It should remain limited to facets absent from runtime metadata. +Maps from typed client method calls to command names are one example. + +Every public command should have at least one provider-backed integration scenario through the request router and request-scoped provider boundaries. +Unit tests remain for: + +- Parser matrices. +- Selector matching. +- Capability maps. +- Malformed inputs. +- State machines. +- Cleanup behavior. +- Provider scope routing. +- Platform error boundaries. + +The temporary migration roadmap is complete and was removed from the repository. +Future work should follow current evidence from the progress script, failing tests, or concrete adapter needs. +It should not follow a permanent refactor backlog. From b8164dfa69fae13904da913d17707d72545fdf8a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:52:54 +0200 Subject: [PATCH 02/55] docs(adr): remove completed ADR 0002 process history --- docs/adr/0002-persistent-platform-helper-sessions.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/adr/0002-persistent-platform-helper-sessions.md b/docs/adr/0002-persistent-platform-helper-sessions.md index 4894081a3f..8044ca7087 100644 --- a/docs/adr/0002-persistent-platform-helper-sessions.md +++ b/docs/adr/0002-persistent-platform-helper-sessions.md @@ -4,6 +4,8 @@ Accepted +Completed process history is in git history. + > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Persistent protocols, > invalidation, and explicitly safe one-shot fallback remain accepted. "Daemon-owned" means the > daemon owns helper lifetime policy; platform modules own process-lived helper managers and helper @@ -62,10 +64,7 @@ the legacy `adb uiautomator dump` snapshot engine. Android permits only one reliable instrumentation-owned `UiAutomation` context per device. Snapshot capture, gesture viewport resolution, and planned-touch injection therefore share one bundled automation helper: a live persistent helper session executes touch commands directly over its -session socket, and without one the same helper runs one-shot (amended 2026-07, issue #1275, -consistent with ADR 0013; previously touch synthesis shipped as a separate instrumentation helper, -so the daemon had to stop the persistent snapshot session before every gesture and let the next -snapshot restart it lazily). One-shot retry after a failed session command applies only to +session socket, and without one the same helper runs one-shot (issue #1275; consistent with ADR 0013). One-shot retry after a failed session command applies only to idempotent reads such as viewport resolution, and only after the failed session has been stopped. Gesture injection is not idempotent — events may already be partially injected — so a session gesture failure surfaces directly instead of retrying one-shot. Helper reuse must never turn From 423fda58fdbd282a9a1357ebffbaefef1561fec7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:53:25 +0200 Subject: [PATCH 03/55] docs(adr): write ADR 0002 in ASD-STE100 --- ...002-persistent-platform-helper-sessions.md | 234 +++++++++++------- 1 file changed, 138 insertions(+), 96 deletions(-) diff --git a/docs/adr/0002-persistent-platform-helper-sessions.md b/docs/adr/0002-persistent-platform-helper-sessions.md index 8044ca7087..8d51be0ef7 100644 --- a/docs/adr/0002-persistent-platform-helper-sessions.md +++ b/docs/adr/0002-persistent-platform-helper-sessions.md @@ -6,115 +6,157 @@ Accepted Completed process history is in git history. -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Persistent protocols, -> invalidation, and explicitly safe one-shot fallback remain accepted. "Daemon-owned" means the -> daemon owns helper lifetime policy; platform modules own process-lived helper managers and helper -> mechanics, and request bindings attach to but never own or stop healthy helper generations. +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Persistent protocols, invalidation, and explicitly safe one-shot fallback remain accepted. +> The daemon owns helper lifetime policy. +> Platform modules own process-lived helper managers and helper mechanics. +> Request bindings attach to healthy helper generations but never own or stop them. -## Context +## Rules at a glance -Some platform automation backends are expensive to start but cheap to reuse. iOS already uses a -long-lived XCTest runner session with an HTTP transport. That model avoids paying `xcodebuild`, -runner boot, and XCTest readiness costs for every command, while still allowing the daemon to -invalidate the runner when the device, app, bundle, or runner process changes. +- When startup costs are high and an automation context is reusable, use persistent platform helper sessions. +- The daemon owns lifetime policy. Platform modules own helper-manager mechanics. +- Unless an explicit cache contract applies, helpers must execute each command against fresh platform state. +- Keep Android snapshots and touch commands in one bundled automation helper. +- For Android, retry one-shot only for idempotent reads after the failed session stops. +- For Android, do not retry gesture injection after a session failure. +- If the packaged Android helper is unavailable or invalid, agent-device must fail closed. -Android snapshot capture initially used a one-shot instrumentation helper. Every snapshot launched -`adb shell am instrument`, connected `UiAutomation`, captured the tree, emitted XML, and exited. -Recent Android snapshot optimizations reduced XML size, idle waiting, extra file I/O, and hidden -content hint work, but a throwaway prototype still showed that process/session startup dominates -steady-state latency: +## Context -- launcher snapshot: one-shot p50 `227ms`, persistent socket p50 `5.8ms` -- React Navigation playground snapshot: one-shot p50 `265.7ms`, persistent socket p50 `16.5ms` +Some platform automation backends have high startup costs but low reuse costs. +iOS already uses a long-lived XCTest runner session with an HTTP transport. +This model avoids repeated `xcodebuild`, runner boot, and XCTest readiness costs for each command. +The daemon can still invalidate the runner when the device, app, bundle, or runner process changes. -The same pressure can appear on new platform adapters. HarmonyOS or other device backends may have -host tools, test runners, accessibility services, or bridge processes with the same shape: expensive -startup, cheap repeated commands, and a need for strict invalidation. +Android snapshot capture initially used a one-shot instrumentation helper. +Each snapshot performed these operations: -## Decision +1. Start `adb shell am instrument`. +2. Connect `UiAutomation`. +3. Capture the tree. +4. Emit XML. +5. Exit. -Use persistent platform helper sessions when a backend has high startup cost and a reusable -automation context. - -A helper session is an optimization layer governed by daemon-owned lifetime policy, not a -replacement for command correctness. Platform modules own helper-manager mechanics. A helper may -keep processes, sockets, runner state, accessibility service flags, or device forwards warm. It must -still execute each command against fresh platform state unless a separate cache contract has explicit -invalidation. - -The session pattern is: - -- start lazily on the first command that benefits from reuse -- bind the session to a device identity and helper/runner identity -- communicate through a small validated protocol with request ids and version metadata -- reuse the session while the identity and protocol remain valid -- invalidate on device disconnect, helper reinstall/version change, process exit, socket/protocol - failure, app/session identity change, or capture options that affect command semantics -- fall back to the existing one-shot path for the current command when reuse fails -- make shutdown best effort and make stale sessions disposable - -For Android snapshots, productize a persistent helper mode that keeps `UiAutomation` alive and -serves fresh snapshot requests over an `adb forward` socket. Do not add snapshot result caching as -part of that first step. The first reliable win is infrastructure reuse, not data reuse. The current -implementation keeps the existing one-shot instrumentation helper as the fallback for startup, -socket, protocol, and request failures. Both transports execute the same packaged helper contract; -agent-device must fail closed when that helper is unavailable or invalid instead of substituting -the legacy `adb uiautomator dump` snapshot engine. - -Android permits only one reliable instrumentation-owned `UiAutomation` context per device. Snapshot -capture, gesture viewport resolution, and planned-touch injection therefore share one bundled -automation helper: a live persistent helper session executes touch commands directly over its -session socket, and without one the same helper runs one-shot (issue #1275; consistent with ADR 0013). One-shot retry after a failed session command applies only to -idempotent reads such as viewport resolution, and only after the failed session has been stopped. -Gesture injection is not idempotent — events may already be partially injected — so a session -gesture failure surfaces directly instead of retrying one-shot. Helper reuse must never turn -process ownership into cross-command interference. - -For iOS, keep the XCTest runner session as the reference implementation for lifecycle and -invalidation behavior. Android does not need to copy iOS internals, but it should reuse the same -lifetime-policy ideas: per-device session management, readiness checks, structured protocol errors, -fallback/invalidation, and request-scoped observability. - -For future platforms such as HarmonyOS, prefer designing adapters around this same helper-session -contract when their native automation layer is runner-like. Avoid embedding platform-specific -startup assumptions directly in command handlers. +Recent Android snapshot optimizations reduced XML size, idle waits, extra file I/O, and work on hidden content hints. +However, a disposable prototype showed that process/session startup dominates steady-state latency: -## Alternatives Considered +- Launcher snapshot: one-shot p50 `227ms`, persistent socket p50 `5.8ms`. +- React Navigation playground snapshot: one-shot p50 `265.7ms`, persistent socket p50 `16.5ms`. -- Keep one-shot helpers only: simplest and robust, but Android measurements show it leaves an order - of magnitude of steady-state snapshot performance on the table. -- Cache snapshots in the daemon: faster for repeated reads, but unsafe after mutations, animations, - navigation, system dialogs, or app process changes unless a mutation generation contract exists. - Cache infrastructure can be added later; it should not be mixed with helper-session reuse. -- Promote an abstract cross-platform runner immediately: tempting, but premature. iOS XCTest, - Android instrumentation, macOS helper, Linux AT-SPI, and future HarmonyOS backends have different - startup and transport mechanics. Share the daemon lifecycle contract first, then extract common - code only where repetition appears. -- Replace Android instrumentation with a normal app service: potentially useful, but Android - `UiAutomation` access is instrumentation-owned. A persistent instrumentation process keeps the - required privilege model while removing repeated process startup. +New platform adapters can have the same costs. +HarmonyOS or other backends may use host tools, test runners, accessibility services, or bridge processes with this pattern: -## Consequences +- High startup costs. +- Low costs for repeated commands. +- A need for strict invalidation. -Persistent helper sessions should be measured before being productized. A prototype or benchmark -should show meaningful wall-clock improvement on a realistic app state, not just a trivial screen. +## Decision -Session managers need more lifecycle tests than one-shot helpers: startup, ready protocol, reuse, -timeout, malformed response, helper version mismatch, device disconnect, install invalidation, -shutdown, exclusive instrumentation handoff, and one-shot fallback. +When a backend has high startup costs and a reusable automation context, use persistent platform helper sessions. + +A helper session is an optimization under daemon-owned lifetime policy. +It does not replace command correctness. +Platform modules own helper-manager mechanics. +A helper may retain processes, sockets, runner state, accessibility service flags, or device forwards for reuse. +Unless a separate cache contract defines explicit invalidation, the helper must execute each command against fresh platform state. + +Use this session pattern: + +1. On the first command that benefits from reuse, start the session lazily. +2. Bind the session to a device identity and helper/runner identity. +3. Communicate through a small validated protocol with request ids and version metadata. +4. While the identity and protocol remain valid, reuse the session. +5. When an invalidation condition occurs, invalidate the session. +6. If reuse fails, use the existing one-shot path for the current command, subject to the retry limits below. +7. Make shutdown best effort. +8. Make stale sessions disposable. + +These conditions invalidate a session: + +- Device disconnect. +- Helper reinstall or version change. +- Process exit. +- Socket or protocol failure. +- App or session identity change. +- Capture options that affect command semantics. + +For Android snapshots, productize a persistent helper mode that retains `UiAutomation` and serves fresh snapshot requests over an `adb forward` socket. +Do not add snapshot result caching in this first step. +Infrastructure reuse provides the first reliable improvement. +The current implementation retains the existing one-shot instrumentation helper as the fallback for startup, socket, protocol, and request failures. +Both transports execute the same packaged helper contract. +If that helper is unavailable or invalid, agent-device must fail closed. +It must not substitute the legacy `adb uiautomator dump` snapshot engine. + +Android permits only one reliable instrumentation-owned `UiAutomation` context per device. +Thus, snapshot capture, gesture viewport resolution, and planned-touch injection share one bundled automation helper (issue #1275; consistent with ADR 0013). +A live persistent helper session executes touch commands directly over its session socket. +Without a live persistent session, the same helper runs one-shot. +After a failed session command, one-shot retry applies only to idempotent reads, such as viewport resolution. +The failed session must stop before this retry. +Gesture injection is not idempotent because events may already be partially injected. +Thus, session gesture failures return directly without a one-shot retry. +Helper reuse must never let process ownership cause interference between commands. + +For iOS, keep the XCTest runner session as the reference implementation for lifecycle and invalidation behavior. +Android does not need to copy iOS internals. +However, it should reuse these lifetime-policy concepts: + +- Per-device session management. +- Readiness checks. +- Structured protocol errors. +- Fallback and invalidation. +- Request-scoped observability. + +For future platforms such as HarmonyOS, prefer this helper-session contract when the native automation layer uses a runner-like model. +Avoid platform-specific startup assumptions in command handlers. -Observability should report whether a command used a persistent session, started one, reused one, -invalidated one, or fell back to one-shot. This keeps CI and user bug reports diagnosable when a -fast path fails. +## Alternatives Considered -Persistent sessions should not make direct interactive commands unexpectedly slow. Use short -connect/request timeouts for the persistent path, then fall back to the existing one-shot timeout -budget. +- **Keep only one-shot helpers.** This is the simplest robust option. + Android measurements show approximately ten times slower steady-state snapshots without reuse. +- **Cache snapshots in the daemon.** This accelerates repeated reads. + Without a mutation generation contract, it is unsafe after mutations, animations, navigation, system dialogs, or app process changes. + Cache infrastructure can be added later. + It should remain separate from helper-session reuse. +- **Create an abstract cross-platform runner immediately.** This is premature. + iOS XCTest, Android instrumentation, macOS helper, Linux AT-SPI, and future HarmonyOS backends have different startup and transport mechanics. + Share the daemon lifecycle contract first. + Extract common code only where repetition appears. +- **Replace Android instrumentation with a normal app service.** This is potentially useful. + However, Android `UiAutomation` access belongs to instrumentation. + A persistent instrumentation process retains the required privilege model and removes repeated process startup. -The daemon remains the owner of helper lifetime policy. Platform modules own helper-manager mechanics -and may expose helper-session operations, but request bindings and command handlers must not directly -own or stop long-lived helper processes or raw host tool state. +## Consequences -This ADR does not require every backend to implement a persistent session. It defines the preferred -shape when the backend has the same startup/reuse economics that iOS and Android snapshots now -demonstrate. +Before persistent helper sessions become product features, they should be measured. +A prototype or benchmark should show meaningful wall-clock improvement on a realistic app state, not only a trivial screen. + +Session managers need more lifecycle tests than one-shot helpers. +These tests cover: + +- Startup. +- Ready protocol. +- Reuse. +- Timeout. +- Malformed response. +- Helper version mismatch. +- Device disconnect. +- Install invalidation. +- Shutdown. +- Exclusive instrumentation handoff. +- One-shot fallback. + +Observability should report whether a command used, started, reused, or invalidated a persistent session, or used a one-shot fallback. +This information helps diagnose CI failures and user bug reports when a fast path fails. + +Persistent sessions should not make direct interactive commands unexpectedly slow. +Use short connect/request timeouts for the persistent path. +Then use the existing one-shot timeout budget for the fallback. + +The daemon remains the owner of helper lifetime policy. +Platform modules own helper-manager mechanics and may expose helper-session operations. +Request bindings and command handlers must not directly own or stop long-lived helper processes or raw host tool state. + +This ADR does not require every backend to implement a persistent session. +It defines the preferred design when a backend has the startup and reuse costs demonstrated by iOS and Android snapshots. From 6ee58c33ceb3a87cade54f8832312e744cabde17 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:53:52 +0200 Subject: [PATCH 04/55] docs(adr): write ADR 0003 in ASD-STE100 --- docs/adr/0003-daemon-command-registry.md | 155 +++++++++++++---------- 1 file changed, 91 insertions(+), 64 deletions(-) diff --git a/docs/adr/0003-daemon-command-registry.md b/docs/adr/0003-daemon-command-registry.md index 4d24859c03..c783754639 100644 --- a/docs/adr/0003-daemon-command-registry.md +++ b/docs/adr/0003-daemon-command-registry.md @@ -4,86 +4,113 @@ Accepted +## Rules at a glance + +- Keep public command identity in `src/command-catalog.ts` and public input/output contracts in `src/commands/**`. +- Keep daemon route ownership and request-policy traits in the daemon command registry. +- Request modules consume named predicates. Handler modules own execution logic without duplicate route coverage tables. +- The daemon registry must not define public grammar, client options, schemas, help, or platform capability support. +- A derived registry is permitted only if daemon-owned declarations, predicate access, public isolation, and compile-time completeness remain intact. + ## Context -Daemon request handling depends on command traits that are not part of the public command surface: -which handler route owns a command, whether tenant lease admission applies, whether session -execution should lock, whether selector validation applies, whether replay can run an action in the -current session scope, whether invalid recordings block the request, whether Android blocking-dialog -recovery applies, and how request-scoped providers resolve a device. +Daemon request behavior depends on traits outside the public command surface: + +- Which handler route owns a command. +- Whether tenant lease admission applies. +- Whether session execution should lock. +- Whether selector validation applies. +- Whether replay can run an action in the current session scope. +- Whether invalid recordings block the request. +- Whether Android blocking-dialog recovery applies. +- How request-scoped providers resolve a device. -Those traits used to be spread across `src/command-catalog.ts`, request-policy modules, and -handler-local coverage tables. That made `src/command-catalog.ts` carry daemon-only behavior next -to public command identity, and it required duplicate command sets to stay aligned by convention. +Previously, `src/command-catalog.ts`, request-policy modules, and handler-local coverage tables each held some of these traits. +Thus, `src/command-catalog.ts` contained daemon-only behavior beside public command identity. +Duplicate command sets required alignment by convention. ## Decision -Keep public command identity in `src/command-catalog.ts` and public input/output contracts in -`src/commands/**`. +Keep public command identity in `src/command-catalog.ts`. +Keep public input/output contracts in `src/commands/**`. + +Add `packages/daemon/src/core/daemon-command-registry.ts` as the daemon-side source of truth for command route ownership and request-policy traits. +Request modules consume predicate functions from the registry instead of duplicate command string sets. +Handler modules own execution logic only. +They do not export duplicate coverage tables to prove route membership. + +The daemon registry is internal-only. +It must not define: -Add `packages/daemon/src/core/daemon-command-registry.ts` as the daemon-side source of truth for command route -ownership and daemon request-policy traits. Request modules consume predicate functions from the -registry instead of recreating command string sets. Handler modules own execution logic only; they do -not export duplicate coverage tables to prove route membership. +- CLI grammar. +- Node.js client options. +- MCP schemas. +- User-facing help. +- Platform capability support. -The daemon registry is internal-only. It must not define CLI grammar, Node.js client options, MCP -schemas, user-facing help, or platform capability support. Those remain owned by the command -contract, projection, help, and capability modules. +The command contract, projection, help, and capability modules retain these responsibilities. ## Alternatives Considered -- Keep daemon groups in `src/command-catalog.ts`: this keeps one command-name file, but it mixes - public command identity with daemon runtime policy and makes the catalog grow for internal-only - routing decisions. -- Keep handler-local coverage tables: this makes each handler self-describing, but creates a second - route membership source that can drift from the router and request-policy modules. -- Put route checks directly in request modules: this is locally simple, but scatters command - classification across admission, locking, provider scoping, replay, recording, and generic - dispatch. +- **Keep daemon groups in `src/command-catalog.ts`.** This retains one file for command names. + However, it mixes public command identity with daemon runtime policy. + It also increases catalog size for internal-only route decisions. +- **Keep handler-local coverage tables.** Each handler describes its routes locally. + However, this creates a second source of route membership that can disagree with the router and request-policy modules. +- **Put route checks directly in request modules.** This is locally simple. + However, command classification becomes distributed across admission, locking, provider scope, replay, recording, and generic dispatch. ## Consequences -Adding or moving a daemon-handled command requires updating the daemon command registry with its -route and request-policy traits. The registry tests pin the trait decisions, while provider-backed -integration scenarios verify important request-policy behavior through the real daemon request path. +When a daemon-handled command is added or moved, its route and request-policy traits require updates in the daemon command registry. +Registry tests fix the expected trait decisions. +Provider-backed integration scenarios verify important request-policy behavior through the real daemon request path. -The registry file is intentionally a dense internal contract. Its interface should stay small: -callers ask daemon-policy questions through named predicates rather than reading or mutating command -sets. +The registry file intentionally concentrates the internal contract. +Its interface should remain small. +Callers ask daemon-policy questions through named predicates instead of direct access to command sets. -`AGENTS.md` should contain only the operating rule and relevant file pointers for agents. This ADR -owns the rationale so future changes do not need to infer it from agent instructions. +`AGENTS.md` should contain only the operating rule and relevant file pointers for agents. +This ADR owns the rationale. +Thus, future changes do not require readers to infer the rationale from agent instructions. ## Update (2026-06): single-declaration / derivation model -A later proposal (the `CommandDescriptor` direction, now [ADR 0008](0008-command-descriptor-registry.md)) unifies a command's -declarations so the public catalog, capability matrix, CLI/MCP projections, batch allowlist, and this -daemon registry are *derived* from one registration site, to remove the cross-table drift that several -of these surfaces are kept aligned against by convention. - -**This ADR's decision stands.** Its boundary is about *ownership* and the *predicate interface*, not -about the physical file a trait is typed in. "Separate source of truth" means separately owned and -exposed through named predicates — a property that survives a projected/derived backing table. A -derived daemon registry is therefore permitted **only if** it preserves all of the following invariants: - -1. **Daemon-owned declaration.** Route and request-policy traits are declared in a daemon-owned facet - (under `packages/daemon/src/`) and *composed* into the registration — never inlined as fields on the public - command contract in `src/commands/**` or `src/command-catalog.ts`. Co-locating a registration *call* - is fine; co-locating *ownership* of daemon policy in the public surface is the contamination this ADR - rejected (see Alternatives, "Keep daemon groups in `src/command-catalog.ts`"). -2. **Predicate interface unchanged.** Consumers keep asking daemon-policy questions through the named - predicates (`getDaemonCommandRoute`, `isLeaseAdmissionExempt`, `shouldLockSessionExecution`, …). The - daemon registry remains their sole exposer; derivation changes how the backing table is *built*, not - how it is *read*. -3. **No leakage into public projections.** The catalog/CLI/MCP/help/capability projections must be - type-prevented from reading daemon-only traits, and the daemon registry must still not define CLI - grammar, Node.js options, MCP schemas, user-facing help, or capability support. -4. **One declaration per concern, enforced by types.** The single registration site must make a missing - or duplicated daemon trait a *compile error* — replacing today's "aligned by convention". This is the - structural improvement that justifies derivation over a separately hand-authored table. - -A single flat public descriptor whose daemon fields leak into public views is **not** permitted — that is -the "collapse daemon policy into a public command registry" failure this ADR exists to prevent. Compose -facets owned by their domains; derive the registry from them. Until that derivation lands and is pinned -by the registry tests, the hand-authored `packages/daemon/src/core/daemon-command-registry.ts` remains the source of -truth and the operating rule in `AGENTS.md` is unchanged. +A later proposal, the `CommandDescriptor` direction in [ADR 0008](0008-command-descriptor-registry.md), unifies a command's declarations at one registration site. +It derives these surfaces from that site: + +- Public catalog. +- Capability matrix. +- CLI/MCP projections. +- Batch allowlist. +- Daemon registry. + +This removes differences between tables that currently require alignment by convention. + +**This ADR's decision remains accepted.** +Its boundary concerns ownership and the predicate interface, not the physical file that contains a trait. +“Separate source of truth” means separate ownership and exposure through named predicates. +A projected or derived table can preserve that property. +A derived daemon registry is therefore permitted **only if** it preserves all these invariants: + +1. **Daemon-owned declaration.** A daemon-owned facet under `packages/daemon/src/` declares route and request-policy traits. + The registration composes that facet. + The public command contract in `src/commands/**` or `src/command-catalog.ts` must never contain these traits as inline fields. + A registration call may be co-located with public declarations. + Ownership of daemon policy must remain separate from the public surface. + See Alternatives, “Keep daemon groups in `src/command-catalog.ts`”. +2. **Predicate interface unchanged.** Consumers continue to use named predicates such as `getDaemonCommandRoute`, `isLeaseAdmissionExempt`, and `shouldLockSessionExecution`. + Only the daemon registry exposes these predicates. + Derivation changes table construction, not table access. +3. **No leakage into public projections.** Types must prevent catalog/CLI/MCP/help/capability projections from access to daemon-only traits. + The daemon registry must still not define CLI grammar, Node.js options, MCP schemas, user-facing help, or capability support. +4. **One declaration per concern, enforced by types.** At the single registration site, a missing or duplicate daemon trait must cause a compile error. + This replaces alignment by convention. + This structural improvement justifies derivation instead of a separate manually authored table. + +A single flat public descriptor that exposes daemon fields in public views is **not** permitted. +This would combine daemon policy with a public command registry, which this ADR rejects. +Compose facets owned by their domains. +Derive the registry from those facets. +Until registry tests verify the implemented derivation, the manually authored `packages/daemon/src/core/daemon-command-registry.ts` remains the source of truth. +The operating rule in `AGENTS.md` remains unchanged until then. From 4ea1d36554c8cbd25364a2862b46e46b40565618 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:54:13 +0200 Subject: [PATCH 05/55] docs(adr): remove completed ADR 0004 process history --- docs/adr/0004-ios-snapshot-backend-strategy.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/docs/adr/0004-ios-snapshot-backend-strategy.md b/docs/adr/0004-ios-snapshot-backend-strategy.md index 9657ecd3d1..fe1b664f56 100644 --- a/docs/adr/0004-ios-snapshot-backend-strategy.md +++ b/docs/adr/0004-ios-snapshot-backend-strategy.md @@ -2,8 +2,9 @@ ## Status -Accepted. Amended after local iOS Simulator acquisition moved to the host AX bridge while the -public surface remained two modes: regular interactive snapshots and raw diagnostic snapshots. +Accepted. + +Completed process history is in git history. The Apple platform runtime owns acquisition routing and its generation-scoped XCTest fallback. Host-side iOS validation, semantic presentation, and publication are owned by @@ -210,8 +211,7 @@ runner does not mute ends `testCommand` as soon as the main-thread block that re whatever `continueAfterFailure` says, so the runner died after (or during) every hostile snapshot and the per-bundle penalty and depth memory died with it. The runner now mutes AX-server rejections in both XCTest fetch wordings, and every bounded main-thread dispatch that outlives its slice counts -as occupying the main thread (the tree XPC and the system-modal probe previously kept a second count -of their own), so a viewport read that grinds makes the plan skip the sweep instead of queueing it. +as occupying the main thread, so a viewport read that grinds makes the plan skip the sweep instead of queueing it. ## Recovery conformance and depth hints @@ -245,9 +245,7 @@ construct `RawAXNode`, `SnapshotPresentation` alone constructs `PresentedNode`, accept only presented nodes. Its second behavior-preserving step makes every capture-plan backend return `SnapshotAcquisition` and routes the exhaustive backend switch through one `SnapshotPresentation.present` call. `PresentationOptions` is the stable request-policy input to -that boundary. Until the remaining migration steps move interpretation into the boundary, acquisition -still reads those options and raw nodes intentionally carry the derived fields produced by the -existing backends. +that boundary. The first semantic migration layer makes regular eligibility backend-neutral inside `SnapshotPresentation`: the top-level viewport carrier survives, and every other node needs an @@ -315,9 +313,7 @@ root clip to violate, and a node whose actionability depends on containment has wire, as on the host bridge; disabled or degenerate nodes stay declared `false`. A rectangle becomes a `Box` only through the initializer that checks it, and the shared guard refuses `CGRect.infinite` by identity — its components and its extents are all finite, so no comparison would have caught the -value a failed read leaves behind. The two twins were not twins before #2908 landed: the Swift guard -already refused that value by identity, while the TypeScript one accepted it and both accepted finite -components whose right or bottom edge overflowed. Both now refuse both shapes, and every box either +value a failed read leaves behind. Both now refuse both shapes, and every box either guard accepted and is neither of those two shapes is still classified the same way. The runner route's host evidence comes from the payload's root nodes (`resolveIosViewportEvidenceFromRoots` in `packages/capture-kit/src/ios-snapshot-acquisition.ts`). `contracts/fixtures/snapshot-actionability-policy.json` From 2d6a749eee0cf86b922765d06ea57c3f08f5f325 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:56:54 +0200 Subject: [PATCH 06/55] docs(adr): write ADR 0004 in ASD-STE100 --- .../adr/0004-ios-snapshot-backend-strategy.md | 1095 +++++++++-------- 1 file changed, 591 insertions(+), 504 deletions(-) diff --git a/docs/adr/0004-ios-snapshot-backend-strategy.md b/docs/adr/0004-ios-snapshot-backend-strategy.md index fe1b664f56..60b53a981a 100644 --- a/docs/adr/0004-ios-snapshot-backend-strategy.md +++ b/docs/adr/0004-ios-snapshot-backend-strategy.md @@ -6,542 +6,629 @@ Accepted. Completed process history is in git history. -The Apple platform runtime owns acquisition routing and its generation-scoped XCTest fallback. -Host-side iOS validation, semantic presentation, and publication are owned by -`@agent-device/capture-kit`; structured snapshot quality verdicts and fallback warnings make -degraded or recovered output observable end to end. +The Apple platform runtime owns acquisition routes and its generation-scoped XCTest fallback. +`@agent-device/capture-kit` owns host-side iOS validation, semantic presentation, and publication. +Structured snapshot quality verdicts and fallback warnings expose degraded or recovered output through the complete path. + +## Rules at a glance + +- For eligible local iOS Simulator snapshots, use host AX acquisition and shared TypeScript presentation. +- If bridge acquisition or presentation fails, use one typed XCTest fallback. +- After an app-specific failure, disable the bridge for that app generation. A new generation enables it again. +- Keep screen-specific coordinate-space refusals scoped to the capture. +- Keep regular visibility and raw diagnostics as explicit public strategies. +- Regular snapshots must not omit visible controls through an arbitrary node-count cutoff. +- If raw output exceeds transport capacity, fail explicitly instead of silently truncating it. +- New host-side consumers must use the snapshot policy facet. +- Acquire raw facts. Apply visibility, eligibility, scope, and regular depth in presentation. +- Observe registered system surfaces in place. Never activate them. ## Context -Agent Device exposes iOS UI state through host AX acquisition on local Simulators and the long-lived -XCTest runner everywhere else. The snapshot surface has two durable needs: +Agent Device exposes iOS UI state through host AX acquisition on local Simulators and the long-lived XCTest runner elsewhere. +The snapshot surface has two durable needs: -- agent-facing regular context, where the important contract is the effective user-visible UI, - fixed controls such as tab bars, and scroll-hidden hints for content outside visible scroll - containers; -- rich diagnostics and selector disambiguation, where a raw recursive XCTest snapshot is useful - because it preserves hierarchy, static text, wrappers, scroll containers, and ancestry. +- Regular context for agents: effective user-visible UI, fixed controls such as tab bars, and hints about content outside visible scroll containers. +- Rich diagnostics and selector disambiguation: raw recursive XCTest snapshots that preserve hierarchy, static text, wrappers, scroll containers, and ancestry. -These needs should not share one capture strategy blindly. Recursive `XCUIElement.snapshot()` is -rich, but some real app trees can make XCTest fail with `kAXErrorIllegalArgument` or main-thread -timeouts while the same app remains visually usable. Bluesky is the current known example: -lower-level accessibility services can describe simulator screens even when XCTest recursive -snapshots and typed `XCUIElementQuery` enumeration degrade to no useful child nodes. Physical iOS -devices can show the same XCTest accessibility-channel timeout shape even when no lower-level -semantic backend is available. +These needs should not use one capture strategy without distinction. +Recursive `XCUIElement.snapshot()` provides rich data. +However, some real app trees cause `kAXErrorIllegalArgument` or main-thread timeouts while the app remains visually usable. +Bluesky is the current known example. +Lower-level accessibility services can describe simulator screens even when recursive XCTest snapshots and typed `XCUIElementQuery` enumeration return no useful child nodes. +Physical iOS devices can have the same XCTest accessibility-channel timeout without an available lower-level semantic backend. -This is different from presentation filtering. The daemon's snapshot presentation can hide noisy -or inaccessible nodes, but it cannot recover nodes that XCTest never returns. More filters, -Maestro-specific heuristics, or retries in the daemon would only make this failure slower and less -predictable. +Presentation filters cannot correct this failure. +The daemon's snapshot presentation can hide noisy or inaccessible nodes. +It cannot recover nodes that XCTest never returns. +More filters, Maestro-specific heuristics, or daemon retries would only make the failure slower and less predictable. ## Decision -Keep XCTest as the iOS automation runner. Route eligible local iOS Simulator snapshots through the -host AX bridge, present them once through the shared TypeScript engine, and use one typed XCTest -fallback when bridge acquisition or presentation fails. Disable the bridge for that app generation -after fallback; a new app generation re-enables it. Physical devices, providers, custom-action -captures, and interactions remain on their existing owners. - -A WebKit page — Safari's, or a `WKWebView`'s — lives in a WebContent process and reaches UIKit's -tree as an `AXRemoteElement` under the web view, with its children in that other process. The -bridge reads one process, so it delivers the element as a leaf. The source refuses a tree in which -such a leaf sits under a `WebView`-typed ancestor and reaches the viewport (`remote-content-boundary`) -instead of publishing a screen without its page: refs issued from it would target the host views -around the page rather than the page. A leaf whose frame is zero-area or off screen hosts nothing -the capture can miss and is published; one that reports no frame is refused, because nothing proves -it empty. A scene-hosted remote view controller — a share or action extension presented over the -host app — reaches the tree the same way: an `AXRemoteElement` leaf under a `_UISceneHostingView`, -with the extension's controls in its own process. The source refuses that leaf by the same rule. -Remote elements under any other host are not classified — no capture has shown one — and a host -truncated away by the node or depth cap stays disclosed as truncation. XCTest resolves remote -elements, so the fallback serves the page or the extension (#2484). - -The refusal opens the generation circuit, as any failure that says something about the app itself -does, so a hybrid app that showed one web screen takes XCTest for its remaining native screens until -it relaunches — the 0.20.x path for every screen. Re-asking the bridge per capture would instead -charge a refused bridge round trip to every `wait` poll on the web screen; the circuit keeps that cost -to one capture per generation. A refusal that says something about the screen rather than the app is -scoped per capture instead; the coordinate-space amendment below decides that case. +Keep XCTest as the iOS automation runner. +Route eligible local iOS Simulator snapshots through the host AX bridge. +Present them once through the shared TypeScript engine. +If bridge acquisition or presentation fails, use one typed XCTest fallback. +After fallback, disable the bridge for that app generation, subject to the capture-scoped exception below. +A new app generation enables it again. +Physical devices, providers, custom-action captures, and interactions retain their existing owners. + +A WebKit page in Safari or a `WKWebView` runs in a WebContent process. +Its connection to UIKit's tree is an `AXRemoteElement` under the web view. +Its children remain in the other process. +The bridge reads one process, so it returns the element as a leaf. +If that leaf has a `WebView`-typed ancestor and reaches the viewport, the source refuses the tree with `remote-content-boundary`. +Otherwise, published refs would target host views around the page instead of the page. +A zero-area or offscreen leaf cannot contain visible content omitted by the capture, so the source publishes it. +If the leaf reports no frame, the source refuses it because no fact proves it empty. + +A scene-hosted remote view controller, such as a share or action extension, uses the same structure. +It presents an `AXRemoteElement` leaf under a `_UISceneHostingView`. +The extension's controls remain in its own process. +The source refuses this leaf under the same rule. +Remote elements under other hosts remain unclassified because no capture has shown them. +If a node or depth cap removes the host, the capture continues to disclose truncation. +XCTest resolves remote elements, so the fallback serves the page or extension (#2484). + +This refusal opens the generation circuit because it provides evidence about the app. +After a hybrid app shows one web screen, its remaining native screens use XCTest until relaunch. +This is the 0.20.x path for every screen. +A bridge attempt on each capture would add a refused round trip to every `wait` poll on the web screen. +The circuit limits this cost to one capture per generation. +A screen-specific refusal instead has capture scope. +The coordinate-space amendment below defines that case. Keep the two public snapshot strategies explicit: -- **Regular visible strategy**: use recursive XCTest snapshots, emit the effective user-visible - tree plus visible ancestors and scroll-hidden hints, and fall back through the capture plan when - XCTest returns sparse output. A node inside a scroll container is user-visible only when it - intersects both the app viewport and the nearest visible scroll container. Offscreen descendants - should be visited to set `hiddenContentAbove` / `hiddenContentBelow`, not emitted as normal - visible nodes. This strategy must not use an arbitrary node-count cutoff: fixed controls that are - later in traversal order, such as bottom tab bars after long lists, are part of the visible UI - contract. -- **Raw diagnostic strategy**: use recursive XCTest snapshots for raw snapshots, diagnostics, and - cases that need hierarchy. Raw output is allowed to be noisy and large; if the transport cannot - carry the response, fail explicitly instead of silently truncating the tree at a hard node count. - If XCTest reports a real AX serialization failure, preserve that error instead of pretending the - UI is empty. +- **Regular visible strategy**: use recursive XCTest snapshots. + Emit the effective user-visible tree, visible ancestors, and scroll-hidden hints. + If XCTest returns sparse output, use the capture plan's fallbacks. + A node inside a scroll container is user-visible only if it intersects the app viewport and nearest visible scroll container. + Offscreen descendants should be visited to set `hiddenContentAbove` / `hiddenContentBelow`. + They should not be emitted as normal visible nodes. + This strategy must not use an arbitrary node-count cutoff. + Fixed controls late in traversal, such as bottom tab bars after long lists, belong to the visible UI contract. +- **Raw diagnostic strategy**: use recursive XCTest snapshots for raw snapshots, diagnostics, and cases that require hierarchy. + Raw output may be noisy and large. + If the transport cannot carry the response, fail explicitly. + Do not silently truncate the tree at a hard node count. + If XCTest reports a real AX serialization failure, preserve the error instead of reporting an empty UI. - **Host AX strategy**: acquire local Simulator trees as raw facts through the bounded host bridge. - Every result crosses the same presentation boundary before publication. XCTest fallback carries - explicit source residue, and comparisons require matching producer, intent, app generation, - presentation key, and residue. Physical devices should use an equivalent non-XCTest semantic - backend only if Apple exposes a supported channel. + Before publication, pass every result through the same presentation boundary. + XCTest fallback carries explicit source residue. + Comparisons require matching producer, intent, app generation, presentation key, and residue. + Physical devices should use an equivalent non-XCTest semantic backend only if Apple exposes a supported channel. -The daemon should make degraded output observable. If an iOS interactive snapshot contains only the -application root or another sparse shape, surface a structured quality verdict and warning so -agents know the snapshot is degraded output rather than proof that the screen has no controls. +The daemon should expose degraded output. +If an iOS interactive snapshot contains only the application root or another sparse shape, return a structured quality verdict and warning. +Thus, agents can distinguish degraded output from proof that a screen has no controls. ## Host-side ownership boundary -The shared TypeScript side has one snapshot-presentation facet. The neutral acquisition-to-presented -carrier and clip-fold geometry contract live in `@agent-device/contracts/snapshot-presentation`, while -`@agent-device/capture-kit` owns host-side iOS planning, folding, projection, eligibility, semantic -compaction, validation, and publication. Platform-specific presentation adapters retain only the -policy mechanics that cannot yet cross their runtime boundary. Daemon assembly owns only the ordering -of capture, compaction, occlusion, and ref publication. It does not own the presentation vocabulary -or a second geometry carrier. - -Android acquisition remains in its platform module and adapts its raw hierarchy to the shared -carrier. Swift keeps its runner-side `SnapshotPresentation` implementation because it consumes the -capture-plan tier before the process boundary. The iOS engine fixture is the shared proof between -those runtimes; it does not imply that Swift and TypeScript share an implementation. -The macOS XCTest runner is the desktop-surface exception: its already-presented nodes bypass the iOS -presentation engine and continue through neutral snapshot assembly. - -The same split now holds for the three remaining Wave 4 policies tracked by #1983, so the host-side -facet in `@agent-device/capture-kit` owns snapshot policy generally rather than presentation alone: - -- **Freshness recovery.** The freshness window, the Android staleness classification and its - thresholds, and the retry loop live in `packages/capture-kit/src/snapshot/snapshot-freshness/`. - The loop is parameterized by a classifier and a retry schedule, so "how long may a backend lag - behind a real transition" is a policy input rather than a constant the loop owns. The schedule is - stated as a duration budget; the loop derives the deadline from the window's `markedAt` itself, so - the budget is always spent from the action and a caller has no absolute instant it could get - wrong. - `packages/daemon/src/core/session-snapshot-freshness.ts` keeps only what needs a session: reading and retiring - the window on store-owned `SessionState`, and choosing the comparison baseline from snapshot - lineage. It remains the declared R7 owner of `androidSnapshotFreshness`. -- **Timeout evidence.** Whether a capture failed because the hierarchy never arrived is decided - once, at the deepest boundary that has the evidence, from machine-defined values only: - `snapshot-capture-failure-reason.ts` maps the helper's structured `errorType` field - (`java.util.concurrent.TimeoutException`, by exact equality) and the SIGKILL exit code 137 to - the typed reason `accessibility-timeout` (`ANDROID_CAPTURE_FAILURE_REASONS` in - `@agent-device/contracts/android-snapshot-quality`). The helper-result, session-protocol, and - killed-instrumentation error constructors attach it; every layer above rewraps it rather than - reclassifying. No message shape is consulted anywhere on that path, so rewording helper or - wrapper prose cannot move the reason, and prose that merely reads like a timeout does not become - one — both directions are asserted end to end against the real producer. - `packages/capture-kit/src/snapshot/snapshot-timeout-policy.ts` reads the reason; the - human-facing hint is derived from it rather than decided alongside it. - - The published `details.androidSnapshotTimeoutScreenshot` payload is vocabulary in - `@agent-device/contracts/snapshot-timeout-evidence`, a union whose arms encode which claims can - coexist. The annotated arm carries a non-empty ref tuple, so "annotated with zero refs" is not a - state a caller can build, and no arm stores a ref count: a count beside the refs is a second - source of truth the type system cannot hold in step, so it is derived from the refs instead. - The daemon keeps the ordering that genuinely needs it: resolving a bound screenshot runtime, - writing the artifact, annotating it from the stored observation, and emitting the diagnostics. -- **Screenshot-overlay policy.** Which Android nodes earn an overlay ref, and what rectangle an - overlay for one of them covers, live in `packages/capture-kit/src/screenshot-overlay*.ts`. The - daemon keeps approved artifact and ref assembly only: ranking, projection to screenshot pixels, - drawing, and PNG IO. - -`scripts/layering/snapshot-presentation-boundary.test.ts` enforces the direction, but only across -the roots `snapshot-policy` declares in `scripts/layering/architecture-ownership.ts`: nothing in -that snapshot tree may import `packages/daemon/src/`, while the overlay modules beside the tree sit outside -those roots and are outside that gate. It carries a positive control, because a filter that stopped -matching would look identical to a boundary being obeyed. - -The residual call sites #1983 also named are audited and deliberately left in place. -`packages/daemon/src/core/direct-ios-selector.ts` carries no presentation policy: `isLocalIosRunnerSession` and -`readSimpleIosSelectorTarget` are session routing (device family, provider ownership, the -stabilization window), while `deriveDirectIosNodeSelector` and `isDirectIosSelectorFallbackError` -are selector derivation and ADR 0011 delegation-on-error. The latter two are pure and -daemon-independent, but their owner would be the selector pipeline governed by R19, not this -facet; moving them under ADR 0004 would widen it to a boundary it does not decide. The -observation and interaction consumers — `selector-capture-runtime.ts`, -`deferred-interaction-outcome.ts`, `snapshot-capture.ts` and -`interaction-touch-android-freshness.ts` — now reach freshness only through the facet or its -session binding. - -New consumers must use the facet rather than add another daemon presentation path. - -The acquisition/presentation boundary has two explicit vocabularies. An acquired input is raw node -evidence accompanied by its capture hint, viewport, lineage, and residue; the host engine folds and -projects that evidence. A presented input is the runner's primary payload plus validation facts; the -host engine validates it and performs semantic compaction once. Regular eligibility decides which -nodes belong in the regular presentation, while publication adds refs and emits only the primary -payload. An optional unscoped quality payload is validated for classification evidence and is never -published. - -Semantic compaction may move an identifier; it may not un-make one. A structural `Other` wrapper -carrying an identifier and nothing else is suppressed in favour of its content, which is a -delegation: the identifier goes on living in whatever the wrapper stood for. A wrapper with no -content has nothing to delegate to, so suppressing it deletes the identifier from every canonical -view while `is`, `get`, and `click` still resolve it from the same capture. That deletion needs the -node's own declared `hittable: false`, because it is the only verdict in the capture that says the -wrapper is inert; a producer that reports no hittability for any node declares nothing, and an -absent fact is not a negative answer (#2638). +The shared TypeScript side has one snapshot-presentation facet. +`@agent-device/contracts/snapshot-presentation` owns the neutral acquisition-to-presented carrier and clip-fold geometry contract. +`@agent-device/capture-kit` owns these host-side iOS responsibilities: + +- Plans. +- Clip fold. +- Projection. +- Eligibility. +- Semantic compaction. +- Validation. +- Publication. + +Platform-specific presentation adapters retain only policy mechanics that cannot yet cross their runtime boundary. +Daemon assembly owns only the order of capture, compaction, occlusion, and ref publication. +It does not own the presentation vocabulary or a second geometry carrier. + +Android acquisition remains in its platform module and adapts its raw hierarchy to the shared carrier. +Swift retains its runner-side `SnapshotPresentation` implementation because it consumes the capture-plan tier before the process boundary. +The iOS engine fixture provides shared proof between these runtimes. +It does not imply a shared implementation between Swift and TypeScript. +The macOS XCTest runner is the desktop-surface exception. +Its already-presented nodes bypass the iOS presentation engine and continue through neutral snapshot assembly. + +The host-side facet in `@agent-device/capture-kit` owns snapshot policy generally, including the three remaining Wave 4 policies tracked by #1983: + +- **Freshness recovery.** `packages/capture-kit/src/snapshot/snapshot-freshness/` owns the freshness window, Android staleness classification, its thresholds, and the retry loop. + A classifier and retry schedule parameterize the loop. + Thus, the permitted backend delay after a real transition is a policy input, not a loop-owned constant. + The schedule states a duration budget. + The loop derives the deadline from the window's `markedAt`. + Thus, the budget starts at the action, and callers cannot supply an incorrect absolute instant. + `packages/daemon/src/core/session-snapshot-freshness.ts` retains only session-dependent operations. + It reads and retires the window on store-owned `SessionState` and selects the comparison baseline from snapshot lineage. + It remains the declared R7 owner of `androidSnapshotFreshness`. +- **Timeout evidence.** The deepest boundary with evidence classifies a hierarchy timeout once, from machine-defined values only. + `snapshot-capture-failure-reason.ts` maps the helper's structured `errorType` and SIGKILL exit code 137 to `accessibility-timeout`. + The `errorType` match requires exact equality with `java.util.concurrent.TimeoutException`. + `ANDROID_CAPTURE_FAILURE_REASONS` in `@agent-device/contracts/android-snapshot-quality` defines the typed reason. + The helper-result, session-protocol, and killed-instrumentation error constructors attach it. + Each higher layer preserves it in its wrapper instead of reclassifying it. + No layer on this path examines message text. + Changes to helper or wrapper prose cannot change the reason. + Text that resembles a timeout does not create that reason. + End-to-end assertions against the real producer verify both properties. + `packages/capture-kit/src/snapshot/snapshot-timeout-policy.ts` reads the reason and derives the human-facing hint from it. + + `@agent-device/contracts/snapshot-timeout-evidence` owns the published `details.androidSnapshotTimeoutScreenshot` vocabulary. + Its union members encode which claims can coexist. + The annotated member carries a non-empty ref tuple, so callers cannot construct an annotation with zero refs. + No member stores a ref count. + A separate count would create a second source of truth that types cannot keep consistent. + Instead, the count is derived from the refs. + The daemon retains this required order: + + 1. Resolve a bound screenshot runtime. + 2. Write the artifact. + 3. Annotate it from the stored observation. + 4. Emit the diagnostics. + +- **Screenshot-overlay policy.** `packages/capture-kit/src/screenshot-overlay*.ts` decides which Android nodes receive an overlay ref and which rectangle the overlay covers. + The daemon retains only approved artifact and ref assembly: rank, projection to screenshot pixels, drawing, and PNG IO. + +`scripts/layering/snapshot-presentation-boundary.test.ts` enforces the dependency direction across roots declared by `snapshot-policy` in `scripts/layering/architecture-ownership.ts`. +Nothing in that snapshot tree may import `packages/daemon/src/`. +Overlay modules beside the tree are outside those roots and outside that gate. +The test has a positive control. +Without it, a filter that stopped matching would appear identical to a respected boundary. + +The residual call sites named by #1983 remain intentionally in place after an audit. +`packages/daemon/src/core/direct-ios-selector.ts` contains no presentation policy. +`isLocalIosRunnerSession` and `readSimpleIosSelectorTarget` route sessions by device family, provider ownership, and stabilization window. +`deriveDirectIosNodeSelector` and `isDirectIosSelectorFallbackError` provide selector derivation and ADR 0011 delegation-on-error. +The latter two are pure and daemon-independent. +However, R19's selector pipeline would own them, not this facet. +A move under ADR 0004 would extend this ADR beyond its boundary. +These observation and interaction consumers access freshness only through the facet or its session binding: + +- `selector-capture-runtime.ts`. +- `deferred-interaction-outcome.ts`. +- `snapshot-capture.ts`. +- `interaction-touch-android-freshness.ts`. + +New consumers must use the facet instead of another daemon presentation path. + +The acquisition/presentation boundary has two explicit vocabularies. +An acquired input contains raw node evidence, capture hint, viewport, lineage, and residue. +The host engine folds and projects that evidence. +A presented input contains the runner's primary payload and validation facts. +The host engine validates it and performs semantic compaction once. +Regular eligibility determines membership in regular presentation. +Publication adds refs and emits only the primary payload. +An optional unscoped quality payload is validated as classification evidence and is never published. + +Semantic compaction may move an identifier but may not delete its identity. +A structural `Other` wrapper with only an identifier is suppressed in favor of its content. +The content receives the identifier through delegation. +If the wrapper has no content, no recipient exists. +Its suppression would delete the identifier from every canonical view while `is`, `get`, and `click` still resolve it from the same capture. +That deletion requires the node's own declared `hittable: false`. +Only this verdict states that the wrapper is inert. +A producer that reports no hittability declares nothing. +An absent fact is not a negative answer (#2638). ## Regression Notes -PR #639 made XCTest AX serialization failures explicit instead of swallowing them as empty -snapshots. That was the correct diagnostic change, but it exposed apps whose accessibility trees -XCTest cannot serialize. - -Later work moved recovery into the regular visible capture plan so healthy apps keep the fast -recursive tree path while degraded app classes can still return bounded, honest output when -fallback tiers are the only available source of visible controls. - -Issue #1105 showed a second failure shape on the same app class: instead of failing fast with -`kAXErrorIllegalArgument`, the recursive tree capture can grind for many seconds on -heavy/animating screens before failing, pushing the chained plan past the runner's main-thread -watchdog and burying the main queue under retries. The plan now carries its umbrella deadline -into the query-sweep and private-AX tiers (later ladder rungs stop when the budget is spent), -and a slow, timed-out, or watchdog-abandoned XCTest-backed capture penalizes the XCTest -accessibility channel for that bundle for a bounded window. Subsequent regular plans derive the -next step from backend traits (`effectiveSnapshotCapturePlan`): when a runnable non-XCTest backend -exists, they defer to that independent tier; when it does not, as on physical iOS devices today, -they run a short XCTest probe instead of the full tree slice so healthy screens can recover without -repeating the hostile-screen grind. The raw diagnostic plan is exempt — it keeps tree-first error -propagation. - -A third shape followed on the same app class once the plan recovered reliably. The query-sweep -tier's 19 `allElementsBoundByIndex` reads each fail with `kAXErrorIllegalArgument`, and XCTest -records every one as a test failure worded `Failed to resolve query: ...`. Any recorded failure the -runner does not mute ends `testCommand` as soon as the main-thread block that recorded it returns, -whatever `continueAfterFailure` says, so the runner died after (or during) every hostile snapshot -and the per-bundle penalty and depth memory died with it. The runner now mutes AX-server rejections -in both XCTest fetch wordings, and every bounded main-thread dispatch that outlives its slice counts -as occupying the main thread, so a viewport read that grinds makes the plan skip the sweep instead of queueing it. +PR #639 made XCTest AX serialization failures explicit instead of returning empty snapshots. +This diagnostic behavior is correct, but it exposes apps whose accessibility trees XCTest cannot serialize. +Recovery belongs in the regular visible capture plan. +Healthy apps retain the fast recursive tree path. +When only fallback tiers can provide visible controls, degraded app classes can return bounded output with explicit limits. + +Issue #1105 identifies another failure in the same app class. +On large or animated screens, recursive capture can take many seconds before failure instead of immediately returning `kAXErrorIllegalArgument`. +This can exceed the runner's main-thread watchdog and fill the main queue with retries. +The plan passes its shared deadline to the query-sweep and private-AX tiers. +After the budget expires, later recovery tiers stop. +A slow, timed-out, or watchdog-abandoned XCTest capture penalizes that bundle's XCTest accessibility channel for a bounded window. +Subsequent regular plans derive their next step from backend traits through `effectiveSnapshotCapturePlan`. +If a runnable non-XCTest backend exists, the plan uses that independent tier. +Otherwise, as on current physical iOS devices, it uses a short XCTest probe instead of the full tree slice. +Thus, healthy screens can recover without another long capture of the failing screen. +The raw diagnostic plan is exempt and retains tree-first error propagation. + +The query-sweep tier has 19 `allElementsBoundByIndex` reads. +Each can fail with `kAXErrorIllegalArgument` on the same app class. +XCTest records each as a test failure worded `Failed to resolve query: ...`. +An unmuted failure ends `testCommand` when its main-thread block returns, regardless of `continueAfterFailure`. +A runner exit also loses the per-bundle penalty and depth memory. +Thus, the runner mutes AX-server rejections in both XCTest fetch wordings. +Each bounded main-thread dispatch that exceeds its slice counts as main-thread occupancy. +If a viewport read takes too long, the plan skips the sweep instead of adding it to the queue. ## Recovery conformance and depth hints -The host AX bridge and the XCTest runner's private AX bridge recover rejected deep requests with -different native representations, ladders, and completeness evidence, and they stay separate -implementations. `contracts/fixtures/ios-ax-recovery-conformance.json` is their shared, executable -recovery contract: each producer replays every case through its own adapter, and the fixture -records per-producer expectations plus the intentional differences, so a change to either recovery -path is measured against the same synthetic native world. Common executable policy is extracted only -where the fixture proves equivalence; a shared engine is not a goal. - -The host source additionally keeps a bounded accepted-depth hint per resolved target generation -and producer. It changes only the native levels the first request asks for, is learned only from a -finished recovery that observed a rejection, expires by hinted-capture count so ordinary screens -probe back to the full depth, and is never shared across apps, generations, or producers. The -route's generation circuit remains the only lifecycle owner. +The host AX bridge and the XCTest runner's private AX bridge retain separate recovery implementations. +They use different native representations, recovery tiers, and completeness evidence for rejected deep requests. +`contracts/fixtures/ios-ax-recovery-conformance.json` is their shared executable recovery contract. +Each producer replays every case through its own adapter. +The fixture records per-producer expectations and intentional differences. +Thus, changes to either path use the same synthetic native evidence. +Extract common executable policy only where the fixture proves equivalence. +A shared engine is not a goal. + +The host source also retains a bounded accepted-depth hint per resolved target generation and producer. +The hint changes only the native levels requested by the first request. +Only a finished recovery that observed a rejection can establish it. +It expires by hinted-capture count, so ordinary screens probe the full depth again. +It is never shared across apps, generations, or producers. +The route's generation circuit remains the only lifecycle owner. ## Consequences -Regular snapshots remain the right tool for agents and Maestro compatibility because they describe -what a user can currently perceive and interact with. Raw snapshots remain the right tool when -hierarchy matters. Both may still fail loudly on XCTest-broken trees; that failure is useful -because retrying the same recursive capture is unlikely to reveal a different tree. - -A future AX-service backend is the correct place to regain Bluesky-class semantic coverage. It -should be added as a platform backend with its own lifecycle, protocol, normalization, timing -metrics, and fallback rules, not as another special case inside the XCTest runner. - -The acquire/present migration begins with a behavior-preserving typed seam: acquisition backends -construct `RawAXNode`, `SnapshotPresentation` alone constructs `PresentedNode`, and response payloads -accept only presented nodes. Its second behavior-preserving step makes every capture-plan backend -return `SnapshotAcquisition` and routes the exhaustive backend switch through one -`SnapshotPresentation.present` call. `PresentationOptions` is the stable request-policy input to -that boundary. - -The first semantic migration layer makes regular eligibility backend-neutral inside -`SnapshotPresentation`: the top-level viewport carrier survives, and every other node needs an -interactive accessibility type or a non-empty label, identifier, or value. Hittability no longer -admits an otherwise ineligible node. Raw membership remains unchanged. This is runner eligibility, -not daemon publication membership; backend-blind daemon compaction retains ownership of its declared -noise suppressions. When eligibility removes a structural wrapper, presentation reparents its -surviving descendants to the nearest surviving ancestor and normalizes their indexes and depths. - -The second semantic layer makes scope a presentation specification rather than an acquisition or -daemon-compaction policy. A trimmed non-empty scope selects the first presentation-preorder match -whose subtree contributes to the requested projection; matching inspects label, identifier, and -value case-insensitively. The selected subtree is re-rooted, depth is applied relative to that root, -and no match publishes an empty healthy projection. Swift and TypeScript implementations are pinned -by `contracts/fixtures/snapshot-scope-policy.json`. -Scoped iOS acquisition stays broad (including when depth is requested) until an adapter can prove a -narrowing hint complete. The daemon never reapplies scope after the wire; Android selects its root -inside its TypeScript presentation and desktop surface runtimes retain their platform projection. - -The third semantic layer splits presentation into two projections and gives acquisition one input. -`SnapshotPresentation.captureHint` derives a `CaptureHint` from the request; backends read the hint, -never `PresentationOptions`. A hint names the projection the acquisition must serve and may narrow -acquisition only where the backend can prove the narrowing complete for that projection: scope and -its relative depth never narrow, raw depth does (raw depth *is* traversal depth), and the raw -projection never carries `interactiveOnly`. `presentRegular` folds visibility, eligibility, scope, -and scroll hints; `presentRaw` is the acquired tree, normalized, with scope and depth applied only -when the request asked for them — so `interactive ⊆ regular ⊆ raw` holds per backend rather than per -backend implementation. `snapshot --raw -i` therefore returns the acquired tree instead of an -interactive-filtered one. - -Two structural rules keep a backend from answering a request with the other projection, the shape -that let a recovered `snapshot --raw` return viewport-pruned nodes labeled raw: the raw diagnostic -plan is derived from `SnapshotBackendKind.supportsRawProjection` rather than hand-listed, so a -backend with no hierarchy to return (the query sweep) cannot be planned for raw; and presentation -compares the requested projection with the hint the acquisition was captured under, dropping that -tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure instead of presenting it under the -requested label. - -The fourth semantic layer moves the clip fold itself into presentation. Acquisition backends are -fact serializers: every traversed node is emitted at raw traversal depth with its reported frame, -and `SnapshotAcquisition` carries the viewport. The fold returns a typed carrier with both values: -`raw.rect` remains runner-internal reported geometry, while regular presentation writes the -carrier's effective rectangle through the existing wire `rect` field; raw projections and direct -single-element reads retain reported geometry. `presentRegular` runs the one visibility -interpreter for every backend — viewport ∩ scroll-container clip, the ancestor projection cursor -(an out-of-clip Cell or scroll container hides descendants whose clamped frames would otherwise -leak back into the viewport), the sub-pixel decoration rule, hidden-content hints booked onto -scroll anchors, and reparenting of survivors with collapsed depth. The fold also narrows the -emitted `hittable` to the clip: nothing outside its clip, and nothing without geometry, is ever -hittable regardless of what a backend reported. Platform differences are a `SnapshotFoldPolicy` -input to the shared algorithm (iOS cursor-projected; macOS/tvOS plain viewport intersection), -never a backend exception. The presentation owner validates every framed regular node against its -cumulative effective clip before constructing `SnapshotPresentation.PresentedNode`; frameless and -degenerate semantic carriers stay -eligible but are never actionable, while raw projection remains exempt by contract. A violation is -a typed `IOS_SNAPSHOT_PRESENTATION_FAILED` capture failure with the named `presentation-failed` -snapshot-quality reason, preserved through recovery and the existing TypeScript verdict/warning -contract. - -Inside the runner the viewport is a declared fact, not a rectangle: `SnapshotViewport` is -`reported(box, interfaceOrientation)`, `derived(box)`, or `missing(reason)`, the cases of the host's -`IosViewportEvidence` (#2891). Only `reported` carries an orientation, so only it can anchor a -rotation in `SnapshotGeometrySpace`. With no box the clip skips, the cumulative-clip invariant has no -root clip to violate, and a node whose actionability depends on containment has no `hittable` on the -wire, as on the host bridge; disabled or degenerate nodes stay declared `false`. A rectangle becomes a -`Box` only through the initializer that checks it, and the shared guard refuses `CGRect.infinite` by -identity — its components and its extents are all finite, so no comparison would have caught the -value a failed read leaves behind. Both now refuse both shapes, and every box either -guard accepted and is neither of those two shapes is still classified the same way. The runner route's -host evidence comes from the payload's root nodes (`resolveIosViewportEvidenceFromRoots` in -`packages/capture-kit/src/ios-snapshot-acquisition.ts`). `contracts/fixtures/snapshot-actionability-policy.json` -pins the predicate for shapes the 320x240 fold fixture cannot reach. - -A regular `--depth` request is a presentation cut, not an acquisition bound. `CaptureHint` keeps raw -traversal depth (`--raw --depth`) separate from regular presented depth, but the recursive tree walk -no longer reads presented depth (or any geometry) while descending: for a regular capture it -enumerates the hierarchy — a runaway node cap bounds the walk (#1105, #1156) — and serializes each -node at reported traversal depth and the frame the platform reported. #2661 also removed the per-node -coordinate space (`geometrySpace` / `parentIsWindow`) that #2612 had threaded through every walker; -the one coordinate-space decision is now a single post-acquisition pass over the flat array -(`SnapshotGeometrySpace.normalized`, run in `captureWithBackend`) that keys on ancestry instead of a -value carried down the stack. That pass is why the earlier visible-depth frontier had to go: the -frontier consulted the shared fold mid-walk, and a post-walk normalization pass cannot feed a decision -the walk has already taken — the fold would read reported geometry, and a turned keyboard band in the -device's native space could be cut at the wrong presented depth before the pass ever ran. With the -frontier gone, no acquisition-time decision reads geometry, the walk is bounded only by raw traversal -depth and the node cap, and the visibility fold and the presented-depth cut both happen inside -`SnapshotPresentation`, on the normalized array, at `maximumDepth`. The frontier only ever engaged -when a presented depth was set (`snapshot --depth N`; `snapshot -i` carries none), so that is the -route whose cost the change could move. Measured there on the recursive tier, 15 warm captures per -cell against `main` on the same simulator: the form (135 raw nodes, 4 or 6 presented) and the -scrolled catalog list (279 raw nodes, 4 or 6 presented) walk their whole raw tree on this branch -where `main` pruned it at the presented depth, and acquisition p50 stays within ±4 % and p95 within -±7 % of `main` with mixed sign. The walk runs over an `XCUIElementSnapshot` tree the platform has -already materialized in one call, which is where acquisition time goes; the node construction the -frontier saved is not measurable on these trees. Screens that recover to private AX are untouched -by it. The bound that remains is the raw node cap. De-duplication drops a repeated node and re-parents its children onto that node's -own parent, so identical rows collapse under one addressable owner instead of splitting a subtree -across two nodes with the same identity. Scoped captures remain broad because depth is relative to the -scope root selected in presentation. - -Backend capability declarations are part of the contract, and they describe how much acquisition -work a regular depth request bounds — never whether the backend may answer it. Every backend -serves a regular `--depth` request because presentation applies the presented-depth cut to -whatever hierarchy was acquired: the recursive tree and private AX both enumerate their hierarchy -and are cut afterwards (`presentation-cut`), and the flat query sweep has only its root and one -presented level (so a cut past depth 1 returns the sweep unchanged, `flat`). Completeness below an -acquisition cap is disclosed the same way it is for an -unscoped capture — through `truncated` and `effectiveDepth` — because a depth-capped regular -capture is a subset of the unscoped one from the same backend. Refusing the request instead -produced no answer at all: a plan pinned or deferred to private AX fell through to the synthetic -sparse root, which the daemon then rejected as a missing viewport (#2403). Raw depth remains -acquisition depth for every backend. - -Acquisition-side limits remain explicit: raw private-AX captures still disclose their bridge-side -node cap, the flat query sweep still drops frameless elements because it has no hierarchy to attach -geometryless semantics to, and the recursive tree still has no raw-depth extension for deep XCTest -trees. Presentation cannot repair any of those acquisition limits. - -When adding new iOS snapshot behavior, maintainers should first decide which strategy owns it. If a -change tries to make regular snapshots fast by dropping visible controls behind a node budget, or -tries to make raw snapshots safe by silently truncating, it is probably crossing strategy -boundaries. +Regular snapshots remain appropriate for agents and Maestro compatibility because they describe what a user can currently perceive and operate. +Raw snapshots remain appropriate when hierarchy matters. +Both may still fail explicitly on trees that XCTest cannot capture. +This failure is useful because another identical recursive capture is unlikely to return a different tree. + +A future AX-service backend is the correct place to restore semantic coverage for apps such as Bluesky. +It should be a platform backend with its own lifecycle, protocol, normalization, timing metrics, and fallback rules. +It should not be another special case inside the XCTest runner. + +The acquisition/presentation boundary preserves behavior through types. +Acquisition backends construct `RawAXNode`. +Only `SnapshotPresentation` constructs `PresentedNode`. +Response payloads accept only presented nodes. +Each capture-plan backend returns `SnapshotAcquisition`. +The exhaustive backend switch routes through one `SnapshotPresentation.present` call. +`PresentationOptions` is the stable request-policy input to that boundary. + +`SnapshotPresentation` makes regular eligibility backend-neutral. +The top-level viewport carrier survives. +Every other node needs an interactive accessibility type or a non-empty label, identifier, or value. +Hittability does not admit an otherwise ineligible node. +Raw membership remains unchanged. +This is runner eligibility, distinct from daemon publication membership. +Backend-neutral daemon compaction retains ownership of its declared noise suppressions. +If eligibility removes a structural wrapper, presentation attaches surviving descendants to the nearest surviving ancestor. +It normalizes their indexes and depths. + +Scope is a presentation specification, not an acquisition or daemon-compaction policy. +A trimmed non-empty scope selects the first presentation-preorder match whose subtree contributes to the requested projection. +The match examines label, identifier, and value without case sensitivity. +The selected subtree receives a new root. +Depth applies relative to that root. +If no match exists, publication returns an empty healthy projection. +`contracts/fixtures/snapshot-scope-policy.json` fixes expectations for Swift and TypeScript implementations. +Scoped iOS acquisition remains broad, including with requested depth, until an adapter can prove a narrowing hint complete. +The daemon never reapplies scope after the wire. +Android selects its root inside TypeScript presentation. +Desktop surface runtimes retain their platform projection. + +Presentation has two projections, and acquisition has one input. +`SnapshotPresentation.captureHint` derives a `CaptureHint` from the request. +Backends read the hint, never `PresentationOptions`. +A hint names the projection that acquisition must serve. +It may narrow acquisition only where the backend can prove completeness for that projection. +Scope and its relative depth never narrow acquisition. +Raw depth does narrow acquisition because it is traversal depth. +The raw projection never carries `interactiveOnly`. +`presentRegular` folds visibility, eligibility, scope, and scroll hints. +`presentRaw` returns the normalized acquired tree. +It applies scope and depth only when the request specifies them. +Thus, `interactive ⊆ regular ⊆ raw` holds per backend, independently of implementation. +`snapshot --raw -i` therefore returns the acquired tree without an interactive filter. + +Two structural rules prevent a backend from returning the wrong projection. +Without them, a recovered `snapshot --raw` could return viewport-pruned nodes labeled raw. +The raw diagnostic plan derives from `SnapshotBackendKind.supportsRawProjection`, not a manual list. +Thus, a backend without hierarchy, such as query sweep, cannot enter a raw plan. +Presentation compares the requested projection with the acquisition's capture hint. +If they differ, it drops that tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure. +It does not present that tier under the requested label. + +The clip fold belongs to presentation. +Acquisition backends serialize facts. +They emit each traversed node at raw traversal depth with its reported frame. +`SnapshotAcquisition` carries the viewport. +The fold returns a typed carrier with reported and effective geometry. +`raw.rect` remains runner-internal reported geometry. +Regular presentation writes the carrier's effective rectangle through the existing wire `rect` field. +Raw projections and direct single-element reads retain reported geometry. + +For every backend, `presentRegular` runs one visibility interpreter with: + +- Viewport ∩ scroll-container clip. +- An ancestor projection cursor. +- The sub-pixel decoration rule. +- Hidden-content hints on scroll anchors. +- Surviving nodes attached to surviving ancestors with collapsed depth. + +The cursor prevents an out-of-clip Cell or scroll container's descendants from reappearing through clamped frames. +The fold also restricts emitted `hittable` to the clip. +Nodes outside their clip or without geometry are never hittable, regardless of backend reports. +Platform differences enter the shared algorithm through `SnapshotFoldPolicy`, never a backend exception. +iOS uses cursor projection. +macOS/tvOS use plain viewport intersection. + +Before constructing `SnapshotPresentation.PresentedNode`, the presentation owner validates each framed regular node against its cumulative effective clip. +Frameless and degenerate semantic carriers remain eligible but never actionable. +Raw projection remains exempt by contract. +A violation produces a typed `IOS_SNAPSHOT_PRESENTATION_FAILED` capture failure with the named `presentation-failed` snapshot-quality reason. +Recovery and the existing TypeScript verdict/warning contract preserve that failure. + +Inside the runner, `SnapshotViewport` is a declared fact with the same cases as host `IosViewportEvidence` (#2891): + +- `reported(box, interfaceOrientation)`. +- `derived(box)`. +- `missing(reason)`. + +Only `reported` carries an orientation, so only it can establish a rotation in `SnapshotGeometrySpace`. +Without a box, the clip is skipped, and the cumulative-clip invariant has no root clip to violate. +A node whose actionability depends on containment has no wire `hittable`, as on the host bridge. +Disabled or degenerate nodes remain explicitly `false`. +A rectangle becomes a `Box` only through the validating initializer. +The shared guard rejects `CGRect.infinite` by identity. +Its components and extents are finite, so numeric comparisons cannot reject the value left by a failed read. +Both guards reject that shape and finite components whose right or bottom edge overflows. +They preserve every other box classification that either guard previously accepted. +The runner route derives host evidence from payload root nodes through `resolveIosViewportEvidenceFromRoots` in `packages/capture-kit/src/ios-snapshot-acquisition.ts`. +`contracts/fixtures/snapshot-actionability-policy.json` fixes the predicate for shapes outside the 320x240 fold fixture's range. + +A regular `--depth` request cuts presentation, not acquisition. +`CaptureHint` separates raw traversal depth (`--raw --depth`) from regular presented depth. +During descent, the recursive tree walk reads neither presented depth nor geometry. +For regular capture, it enumerates the hierarchy under a runaway node cap (#1105, #1156). +It serializes each node at reported traversal depth with its platform-reported frame. +A single post-acquisition pass over the flat array decides coordinate space from ancestry. +`SnapshotGeometrySpace.normalized` runs in `captureWithBackend` (#2661). +It replaces the per-node `geometrySpace` / `parentIsWindow` values from #2612. + +A post-acquisition normalization pass cannot supply normalized geometry to an earlier traversal decision. +Thus, a visible-depth frontier that consults the shared fold during traversal would read reported geometry. +It could cut a rotated keyboard band in native device space at the wrong presented depth. +No acquisition-time decision therefore reads geometry. +Only raw traversal depth and the node cap bound the walk. +Inside `SnapshotPresentation`, visibility fold and presented-depth cut use the normalized array at `maximumDepth`. + +The earlier frontier applied only with presented depth, through `snapshot --depth N`. +`snapshot -i` carries no presented depth. +Thus, only the former route could change cost after its removal. +Measurements used the recursive tier, 15 warm captures per cell, and `main` on the same simulator: + +- Form: 135 raw nodes, 4 or 6 presented. +- Scrolled catalog list: 279 raw nodes, 4 or 6 presented. +- Acquisition p50: within ±4 % of `main`, with mixed sign. +- Acquisition p95: within ±7 % of `main`, with mixed sign. + +The changed walk visits the full raw tree where `main` pruned at presented depth. +The platform already materializes the `XCUIElementSnapshot` tree in one call, which dominates acquisition time. +The saved node construction has no measurable cost on these trees. +Screens that recover to private AX remain unaffected. +The raw node cap remains the bound. +De-duplication drops a repeated node and attaches its children to that node's parent. +Thus, identical rows have one addressable owner instead of a subtree split across duplicate identities. +Scoped captures remain broad because depth is relative to the scope root selected in presentation. + +Backend capability declarations describe how much acquisition work a regular depth request bounds. +They never determine whether a backend may answer it. +Every backend serves regular `--depth` requests because presentation cuts the acquired hierarchy. +Recursive tree and private AX enumerate their hierarchy before the cut (`presentation-cut`). +Flat query sweep has only its root and one presented level. +A cut beyond depth 1 leaves the sweep unchanged (`flat`). +`truncated` and `effectiveDepth` disclose completeness below an acquisition cap, as for unscoped capture. +A depth-capped regular capture is a subset of the same backend's unscoped capture. +Refusal previously left a plan pinned or deferred to private AX with only a synthetic sparse root. +The daemon then rejected it for a missing viewport (#2403). +For every backend, raw depth remains acquisition depth. + +Acquisition-side limits remain explicit: + +- Raw private-AX captures disclose the bridge-side node cap. +- Flat query sweep drops frameless elements because it has no hierarchy for geometryless semantics. +- Recursive tree has no raw-depth extension for deep XCTest trees. + +Presentation cannot repair these acquisition limits. + +Before new iOS snapshot behavior is added, maintainers should decide which strategy owns it. +A change probably crosses strategy boundaries if it drops visible controls for a regular node budget or silently truncates raw snapshots. ## Amendment: in-place system surfaces (issue #2438) -Some UI is presented out of the app's process by a system bundle — `com.apple.SafariViewService`, -which hosts `ASWebAuthenticationSession` and `SFSafariViewController` for delegated OAuth/OIDC -sign-in. Two facts, both verified live on the iOS 26.2 Simulator, shape how it is captured: - -- The surface dies if activated. `XCUIApplication.activate()` or `simctl launch` on the host cancels - the authentication session and blacks the view. So the host must be observed and driven **in - place**, never activated, and `open` refuses to launch a registered host. -- The local host AX bridge cannot see it. While the sheet is up the app remains the AX `primaryApp`, - so the bridge serves the (occluded) app tree as if healthy. Only the XCTest runner, addressing the - host by bundle id, can read and drive the sheet. - -Decision. A closed registry names these hosts (`contracts/fixtures/ios-system-surface-hosts.json`, -mirrored by the TypeScript and Swift registries under a parity test). When a registered host is -genuinely presented, the runner serves and drives it in place and never adopts it as the cached -session target; the session binding stays on the app, so once the surface is gone the next command -resolves back to the app. On the Simulator a cheap, device-scoped host-side probe (a registered -host process running for the device) routes the capture to the runner instead of the bridge; when no -host is running the bridge fast path is untouched. - -A presented surface also outranks an explicitly requested bundle id: the runner checks for a -presented host before it resolves or activates `command.appBundleId`, so a command that names a -*different* app is still served the sheet. That is deliberate — the sheet occludes the screen, so -the named app has nothing readable under it, and the capture discloses which surface it describes — -and it costs nothing once the sheet is gone, because the session binding never moved. - -Presence is `XCUIApplication.state == .runningForeground`, not tree content. The live spike showed a -torn-down host still serving a *richer* tree than a live one, so content heuristics cannot separate -live from dead; foreground state can. Crucially, the only way a host is foreground with a stale tree -is if it was activated or relaunched — which the open guard and the in-place policy both refuse — so -this fix and the never-activate guard are one design: the guard is what makes the foreground -predicate sound. This also makes issue #2438's second bug (a stale tree served confidently after -teardown) unrepresentable for the delegated-auth flow, because the session never binds to the host. - -Captures of a system surface carry a response-level `systemSurface` provenance and the shared -`iosSystemSurfaceDisclosure`, worded per host kind, so the agent is told the controls belong to a -system sheet (web sign-in, Apple Pay) rather than the app. They are also lineaged to the host rather -than the app, so their comparison identity differs from an app capture's by construction: every -consumer that asks "are these two captures the same presentation" refuses a cross-surface pair -through ordinary key equality, and no comparison site carries a surface check of its own. Physical -devices always use the runner, so the in-place serve applies there without a route change; the -Simulator route probe is the only Simulator-specific piece. - -The Apple Pay host (`com.apple.PassbookUIService`) joined the registry for text entry as much as for -snapshots. Its billing, shipping, and contact forms hold text fields the session app's tree cannot -resolve, and a bare `type` addressed to the app process never sees that keyboard. Addressing the -host in place is what lets the runner's first-responder route type into them; no text-entry branch -changed for it. +Some UI appears outside the app's process in a system bundle. +For delegated OAuth/OIDC sign-in, `com.apple.SafariViewService` hosts `ASWebAuthenticationSession` and `SFSafariViewController`. +Two facts verified live on the iOS 26.2 Simulator determine capture behavior: + +- **Activation destroys the surface.** `XCUIApplication.activate()` or `simctl launch` on the host cancels the authentication session and makes the view black. + Thus, the host must be observed and operated **in place**, never activated. + `open` refuses to launch a registered host. +- **The local host AX bridge cannot see the surface.** While the sheet appears, the app remains the AX `primaryApp`. + The bridge returns the occluded app tree as healthy. + Only the XCTest runner can read and operate the sheet through the host's bundle id. + +Decision. A closed registry names these hosts in `contracts/fixtures/ios-system-surface-hosts.json`. +TypeScript and Swift registries mirror it under a parity test. +When a registered host is genuinely presented, the runner serves and operates it in place. +It never adopts the host as the cached session target. +The session binding remains on the app. +After the surface disappears, the next command resolves to the app again. +On the Simulator, a low-cost device-scoped host-side probe detects a registered host process for the device. +It routes capture to the runner instead of the bridge. +Without a running host, the bridge fast path remains unchanged. + +A presented surface also takes precedence over an explicitly requested bundle id. +Before the runner resolves or activates `command.appBundleId`, it examines whether a host is presented. +Thus, even a command for a different app receives the sheet. +This is intentional because the sheet occludes the screen and leaves no readable content from the named app. +The capture discloses which surface it describes. +After the sheet disappears, no extra cost remains because the session binding never changed. + +Presence means `XCUIApplication.state == .runningForeground`, independently of tree content. +The live experiment showed that a stopped host could return a richer tree than a live host. +Thus, content heuristics cannot distinguish live and stopped hosts, but foreground state can. +A host can be foreground with a stale tree only after activation or relaunch. +The open guard and in-place policy both refuse those operations. +The never-activate guard therefore makes the foreground predicate valid. +The session never binds to the host. +Thus, delegated-auth flow cannot retain a stale host tree after teardown, the second bug in issue #2438. + +System-surface captures carry response-level `systemSurface` provenance and shared `iosSystemSurfaceDisclosure`, worded for each host kind. +This tells the agent that controls belong to a system sheet, such as web sign-in or Apple Pay, instead of the app. +Their lineage identifies the host instead of the app. +Thus, their comparison identity differs from an app capture's identity by construction. +Ordinary key equality rejects cross-surface comparisons for every consumer. +Comparison sites need no separate surface checks. +Physical devices always use the runner, so in-place behavior applies without a route change. +Only the route probe is Simulator-specific. + +The Apple Pay host, `com.apple.PassbookUIService`, belongs to the registry for text entry as well as snapshots. +Its billing, shipping, and contact forms contain text fields absent from the session app's tree. +A bare `type` addressed to the app process cannot see that keyboard. +The runner's first-responder route can type into those fields when it addresses the host in place. +This requires no changed text-entry branch. ## Amendment: the coordinate space of a captured subtree (issue #2612) -Some system surfaces keep their geometry in the device's native (portrait-up) space while the app -is rotated, and their *whole subtree* arrives turned with them. Measured on iPhone 17 Pro (iOS -26.2) with the system keyboard up over a landscape app frame `(0,0,874,402)`: -`UIRemoteKeyboardWindow` reports its own box as `(0,0,402,874)` — the app's box with its two side -lengths swapped — while the app's own window and `UITextEffectsWindow` both report `(0,0,874,402)`. -In portrait the two spaces coincide and nothing is turned. Nothing but that box says which space a -subtree reports in, and a consumer reading the numbers cannot tell a keyboard laid across the -bottom of a landscape screen from a strip running down its left edge: it refused app content the -keyboard was nowhere near and let a tap land on a key. - -Decision. One published capture publishes one space — the app's orientation space — and the -producer settles it where the platform's frame is still the platform's, because downstream of the -capture the question is unanswerable: a merged tree carries no axis a reader could turn geometry -against. A window declares the space of its own subtree from its own box, which is either the app's -box or that box turned through a quarter, and anything that is not a window inherits the space of -the window above it. A published rect and the actionability verdict read from it come off the same -oriented frame, so an address and the permission to tap it cannot disagree. Where no space can be -named — no usable app frame, an interface orientation the platform did not report, or an app frame -square enough to be indistinguishable from its own quarter turn — the capture publishes what the -platform reported rather than guessing at a rotation, and consumers fail open on that geometry the -way they do on any missing platform fact. Detection and the way back share one rotation table with -synthesized dispatch — pure geometry that lives in the `AgentDeviceSnapshotPresentation` package -(`SnapshotCoordinateSpace.swift`), not in the XCTest bundle — and both languages replay it against -`contracts/fixtures/window-coordinate-space.json` in the ADR 0011 parity-table shape, which is what -stops capture from turning back with anything other than the exact inverse of what dispatch turns -forward. - -Decision. A producer that cannot name the app's interface orientation refuses the screen rather -than publishing two spaces in one tree. The Simulator AX bridge cannot name it: the #2659 spike -(verdict on that issue, write-up in the diff of #2667) measured that the one AX attribute for it, -`XC_kAXXCAttributeApplicationOrientation` (id 1503), resolves but reads 0 through the guest's -snapshot channel and errors `kAXErrorServerNotFound` through XCTest's own reader, and that the only -cheap service read is *device* orientation, which diverges from the app's on a rotation-locked app. -So its decoder counts window roots reporting the app's box quarter-turned and -fails that capture (`window-coordinate-space-unresolved`, kind `unsupported`) — the same refusal -shape as `remote-content-boundary` above — and the route serves the runner, which reads the -orientation. A mixed tree is not half-usable: a published rect is an address, and once some rects -in a tree answer to a turned axis no pair of them answers "how far apart are these" any more, so -every consumer downstream would have to know which windows to distrust — a rule the capture could -have applied and did not. The refusal costs one runner round trip and is correct. - -Decision. That refusal is scoped to the capture, not to the app generation. Entering the generation -circuit is for failures that are evidence about the app: a web-view screen is a property of a -hybrid app, so its screens keep the runner until it relaunches. A rotated surface is evidence about -the screen in front of the reader and about nothing else — it is up now and gone after the next -keystroke — so retiring the generation would move every later portrait capture of a healthy app -onto the runner to work around one landscape keyboard, the cost #2491 settled for a bridge that was -merely still being prepared. Which side of that line a failure falls on is a declared property of -its code rather than a judgment made at the call site. - -What stays unnormalized, and why the consumer rule stays: the runner's recursive-tree capture now -answers the keyboard with a measured band instead of a rebuilt one, because the same run that owns -the tree can ask `app.keyboards` and gets an answer in the app's own orientation space (#2660). That -closes the keyboard question on the path that used to lean on this ADR's geometry most, and it is why -the guard's landscape refusals no longer depend on whether the tree arrived turned. - -What did not move is the rest of the sentence. Capture settles the space in ONE pass over the flat -acquired array (`SnapshotGeometrySpace.normalized`, run once in `captureWithBackend` between -acquisition and presentation, #2661), keyed on each node's `type`, `parentIndex` and `rect`. The -runner's query-sweep tier still has no window ancestry, so nothing in its tree declares a native space: -its flat children hang off a synthetic application root that reports the app's own box under an -interface orientation the tier does not read, so the pass returns that tree exactly as reported — a -structural consequence of the same rule applied uniformly, not the per-tier omission a hand-threaded -space would have made it. The provider producers (`appium-source`, `limrun-ios-tree`) never see the -app's windows either; a capture from any of them publishes no keyboard fact, and a consumer reads that -silence as "this producer did not measure". Those paths publish what the platform reported, so the -last reader that can still refuse geometry it cannot place is the tap-path keyboard guard, and its -width rule therefore remains. The rule detects un-normalized -arrival, not a standing fact about iOS: the producers above do normalize, and a band taller than it -is wide is what one that did not looks like. +Some system surfaces report geometry in the device's native portrait-up space while the app is rotated. +Their whole subtree uses that space. +Measurements used iPhone 17 Pro, iOS 26.2, and the system keyboard over a landscape app frame `(0,0,874,402)`. +`UIRemoteKeyboardWindow` reported `(0,0,402,874)`, with the app's side lengths exchanged. +The app window and `UITextEffectsWindow` both reported `(0,0,874,402)`. +In portrait, the spaces coincide and no rotation occurs. +Only the box identifies the subtree's reported space. +From the numbers alone, consumers cannot distinguish a landscape bottom keyboard from a strip down the left edge. +This caused refusals of uncovered app content and permitted a tap on a key. + +Decision. One published capture uses one space: the app's orientation space. +The producer resolves space while the platform frame still has its original context. +After capture, a merged tree has no axis for consumers to rotate against. +A window's own box declares its subtree's space. +That box is either the app's box or its quarter turn. +Non-window nodes inherit space from the window above them. +Published rectangles and their actionability verdicts use the same oriented frame. +Thus, an address and permission to tap it cannot disagree. + +When space cannot be identified, the capture publishes platform-reported geometry without an assumed rotation. +This applies when: + +- No usable app frame exists. +- The platform does not report interface orientation. +- The app frame is too square to distinguish from its quarter turn. + +Consumers fail open on that geometry, as for other missing platform facts. +Detection and inverse normalization share one rotation table with synthesized dispatch. +This pure geometry belongs to `AgentDeviceSnapshotPresentation` in `SnapshotCoordinateSpace.swift`, outside the XCTest bundle. +Both languages replay `contracts/fixtures/window-coordinate-space.json` in ADR 0011's parity-table shape. +This ensures that capture applies the exact inverse of the forward dispatch rotation. + +Decision. A producer that cannot identify the app's interface orientation refuses the screen instead of publishing two spaces in one tree. +The Simulator AX bridge cannot identify that orientation. +The #2659 experiment, with verdict in that issue and details in #2667's diff, measured the only AX attribute for it. +`XC_kAXXCAttributeApplicationOrientation` (id 1503) resolves but reads 0 through the guest snapshot channel. +Through XCTest's reader, it returns `kAXErrorServerNotFound`. +The only low-cost service read gives device orientation, which differs from app orientation in a rotation-locked app. + +The decoder therefore counts window roots that report the app's box quarter-turned. +It fails that capture with `window-coordinate-space-unresolved`, kind `unsupported`. +This uses the same refusal shape as `remote-content-boundary`. +The route then serves the runner, which reads orientation. +A mixed-space tree cannot provide reliable partial geometry. +Each published rectangle is an address. +If some rectangles use a rotated axis, consumers cannot compare distances between arbitrary rectangles. +Every downstream consumer would need a rule to distrust selected windows. +Capture has the evidence to apply that rule itself. +The refusal costs one runner round trip and is correct. + +Decision. This refusal has capture scope, not app-generation scope. +The generation circuit applies to failures that provide evidence about the app. +A web-view screen identifies a hybrid app, so its screens retain the runner until relaunch. +A rotated surface provides evidence only about the current screen. +It can disappear after the next keystroke. +If it retired the generation, every later portrait capture of a healthy app would use the runner because of one landscape keyboard. +Issue #2491 rejected that cost for a bridge still in preparation. +The failure code declares its scope. +Call sites do not decide it independently. + +The runner's recursive-tree capture measures the keyboard band instead of reconstructing it. +The same run can query `app.keyboards` and receive geometry in the app's orientation space (#2660). +This resolves the keyboard question on the path most dependent on this ADR's geometry. +The guard's landscape refusals therefore no longer depend on whether the tree arrived rotated. + +Capture still resolves space in one pass over the flat acquired array. +`SnapshotGeometrySpace.normalized` runs once in `captureWithBackend`, between acquisition and presentation (#2661). +It uses each node's `type`, `parentIndex`, and `rect`. +The runner's query-sweep tier has no window ancestry, so its tree declares no native space. +Its flat children attach to a synthetic application root with the app's box. +The tier does not read interface orientation. +Thus, the uniform rule returns that tree exactly as reported. +This is a structural consequence, not a tier-specific exception. + +Provider producers `appium-source` and `limrun-ios-tree` also do not see app windows. +Their captures publish no keyboard fact. +Consumers interpret absence as a producer that did not measure the keyboard. +These paths publish platform-reported geometry. +The tap-path keyboard guard is the last consumer able to refuse geometry it cannot place. +Thus, its width rule remains. +The rule detects unnormalized input, not a universal iOS property. +The producers described above do normalize geometry. +A band taller than it is wide indicates a producer that did not normalize it. ## Amendment: modal containment in the presentation cut -The bridge reads one window and reports every container hanging off it, while XCTest's own queries -answer only the presentation the user can reach. React Navigation's card-plus-modal example is the -measured case: three routes presented as sheets left three transition-view containers in one window -and one capture published 567 nodes across three screens, where the same app state under the runner -published 76 across the top screen alone. Maestro's visibility test is geometric, so a `tapOn` -resolved the covered screen's button and the flow failed on an assertion about the screen that never -arrived, and a `presentation: "formSheet"` route's fields matched intermittently (#2638). - -Decision. The fold applies modal containment: when the last transition view under a container carries -UIKit's dimming view as its direct child and the producer reports that dimming view takes touches -(`userInteractionEnabled: true`), the earlier transition views whose frame that dimmed area spans are -cut, with their subtrees, from the regular and interactive projections. Every part of the claim is a -producer fact rather than an ordering guess — the dimming view is UIKit's own declaration that it dims -what sits behind, its interaction state says whether a touch there reaches what sits behind, and its -frame is the producer's rectangle, which is what a covered container must be inside. A presentation -with no dimming view, a dimming view the producer did not read or reports passing touches through, -and any container whose earlier sibling is not a transition view keep today's behavior: containment -is asserted only where the producer states it, so the rule fails closed rather than guessing which -siblings are shadows. - -A sheet resting at an undimmed detent (`largestUndimmedDetentIdentifier`, react-native-screens -`sheetLargestUndimmedDetentIndex`) is why the interaction state is part of the claim. UIKit keeps the -dimming view in the tree at the same window-sized frame, and the presenting screen stays reachable: on -react-navigation's form-sheet example a press on the presenting screen's `Height Steps` button -navigates while the `Custom Dimming` sheet rests at its smallest detent. The bridge reports that -dimming view `userInteractionEnabled: false`, and `true` for the dimmed form sheet and the card modal. -No attribute the bridge already read told them apart, and `XC_kAXXCAttributeIsVisible` cannot either: -it reads false for the undimmed sheet's dimming view and for the card modal's. The bridge reads the -attribute for dimming views alone, in one follow-up read per view: requesting it on every node cost -about half again the capture time on a 492-node tree, while the targeted read did not move it. -Every source the cut removes is counted in the presentation's -`stats.modalContainedNodeCount` — internal evidence, per ADR 0026, never wire vocabulary — because -the same screen either side of an animation otherwise moves hundreds of comparable lines with nothing -to attribute them to. - -Why this seam and not the neighbouring ones. The semantic presentation rules -(`IOS_PRESENTATION_RULES`) run only for the interactive projection, so a suppression rule there could -not deliver the same claim to a regular `snapshot`; the regular eligibility table is pinned against -its Swift twin, so widening it would move a cross-language contract that this rule does not touch; the -occlusion pass *marks* a node covered and keeps it published, and a published-but-marked node is -exactly the 567-versus-76 divergence being fixed, since visibility filtering reads geometry and not -the mark; and cutting at acquisition would leak the decision into `--raw`, which owes the reader the -tree the platform reported. Raw therefore still carries the covered screens. - -Producers that report no UIKit class names or no dimming-view interaction state never trigger the -cut, and that is a fact about their output rather than a backend exception in the fold: the runner already omits modal-contained content -from its own queries, `appium-source` and `limrun-ios-tree` report element types and no classes, and -the macOS desktop surface arrives already presented. A scope naming a modal-contained screen now -publishes an empty projection, which is the same healthy empty answer any other unmatched scope gives -— the screen is not what is presented. +The bridge reads one window and reports all attached containers. +XCTest queries return only the presentation available to the user. +React Navigation's card-plus-modal example provides measured evidence. +Three routes presented as sheets left three transition-view containers in one window. +One bridge capture published 567 nodes across three screens. +In the same app state, the runner published 76 nodes from only the top screen. +Maestro tests visibility geometrically. +Thus, `tapOn` resolved a covered screen's button, and the flow failed an assertion about a screen that never appeared. +A `presentation: "formSheet"` route's fields also matched intermittently (#2638). + +Decision. The fold applies modal containment under all these conditions: + +- A container's last transition view has UIKit's dimming view as a direct child. +- The producer reports that the dimming view takes touches (`userInteractionEnabled: true`). +- The dimmed area spans the frame of an earlier transition view. + +It removes those earlier transition views and their subtrees from regular and interactive projections. +Each condition is a producer fact, not an assumption about order. +UIKit's dimming view declares what it dims. +Its interaction state states whether a touch reaches content behind it. +Its frame supplies the rectangle that must contain the covered container. +These cases retain existing behavior: + +- No dimming view exists. +- The producer did not read the dimming view. +- The dimming view passes touches through. +- The earlier sibling is not a transition view. + +Containment applies only where the producer declares it. +The rule fails closed instead of guessing which siblings are covered. + +A sheet at an undimmed detent explains the interaction-state condition. +The relevant controls are `largestUndimmedDetentIdentifier` and react-native-screens `sheetLargestUndimmedDetentIndex`. +UIKit retains the dimming view with its window-sized frame, while the presenting screen remains reachable. +In react-navigation's form-sheet example, `Height Steps` on the presenting screen navigates while `Custom Dimming` rests at its smallest detent. +The bridge reports `userInteractionEnabled: false` for that dimming view. +It reports `true` for the dimmed form sheet and card modal. +No previously read attribute distinguishes them. +`XC_kAXXCAttributeIsVisible` also cannot distinguish them. +It reads false for both the undimmed sheet's dimming view and the card modal's dimming view. + +The bridge reads interaction state only for dimming views, with one subsequent read per view. +A read on every node added about half the capture time on a 492-node tree. +The targeted read did not change capture time. +Presentation counts every removed source in `stats.modalContainedNodeCount`. +Per ADR 0026, this is internal evidence, never wire vocabulary. +Without it, the same screen before and after animation can change hundreds of comparable lines without attribution. + +Semantic presentation rules (`IOS_PRESENTATION_RULES`) apply only to the interactive projection. +A suppression rule there could not provide the same behavior for regular `snapshot`. +The regular eligibility table has parity with its Swift equivalent. +An extension there would change a cross-language contract outside this rule's scope. +The occlusion pass marks covered nodes but retains them in publication. +Visibility filters read geometry, not that mark. +Thus, published-but-marked nodes retain the 567-versus-76 divergence. +A cut during acquisition would affect `--raw`, which must preserve the platform-reported tree. +Raw therefore retains covered screens. +These constraints place modal containment in the presentation cut. + +Producers without UIKit class names or dimming-view interaction state never trigger the cut. +This follows from their output facts, not backend exceptions in the fold. +The runner already omits modal-contained content from its queries. +`appium-source` and `limrun-ios-tree` report element types without classes. +The macOS desktop surface arrives already presented. +A scope that names a modal-contained screen publishes an empty projection. +This is the same healthy empty result as another unmatched scope because the named screen is not presented. From 9cc762caf7e7ca04e87d4355366e18f46e919f2c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:57:44 +0200 Subject: [PATCH 07/55] docs(adr): remove completed ADR 0005 process history --- docs/adr/0005-ios-runner-interaction-lifecycle.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/adr/0005-ios-runner-interaction-lifecycle.md b/docs/adr/0005-ios-runner-interaction-lifecycle.md index 68f6e0ec2e..799e66541c 100644 --- a/docs/adr/0005-ios-runner-interaction-lifecycle.md +++ b/docs/adr/0005-ios-runner-interaction-lifecycle.md @@ -4,6 +4,8 @@ Accepted +Completed process history is in git history. + ## Context The iOS runner is a long-lived XCTest process with an HTTP command loop. A command can appear to @@ -41,7 +43,7 @@ same `appBundleId` within 5 seconds. This recency lives only on the `RunnerSessi `lastHealthyMutation`, so it dies with every invalidation/restart, and it is recorded only after the `runnerFatal` check, so sparse AX-fallback snapshots and `runnerFatal` payloads never refresh it. Snapshots and other read-only responses never count as a health signal. This narrow skip is -permitted now because the future-work precondition below is met: coordinate-first activation removed +permitted because: coordinate-first activation removed the command-induced teardown trigger, and the lifecycle status journal plus the status-before- invalidate recovery is the teardown-surviving status surface that resolves any ambiguous post-send failure before invalidation. A transport failure after a skip clears the recency record and is marked @@ -147,6 +149,3 @@ an app target across process identity. The reset adds one local runner request i runner restart and clears the old process's hostile-screen capture penalty before the replacement is reacquired. -Future optimization work should only reduce these preflights after the runner exposes status in a -way that survives command-induced XCTest teardown and can prove the session is still serving new -requests. From 685b340b06f9e684f73225e19e3b2512c4c98567 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:57:44 +0200 Subject: [PATCH 08/55] docs(adr): write ADR 0005 in ASD-STE100 --- .../0005-ios-runner-interaction-lifecycle.md | 296 ++++++++++-------- 1 file changed, 164 insertions(+), 132 deletions(-) diff --git a/docs/adr/0005-ios-runner-interaction-lifecycle.md b/docs/adr/0005-ios-runner-interaction-lifecycle.md index 799e66541c..c23efc840e 100644 --- a/docs/adr/0005-ios-runner-interaction-lifecycle.md +++ b/docs/adr/0005-ios-runner-interaction-lifecycle.md @@ -6,146 +6,178 @@ Accepted Completed process history is in git history. +## Rules at a glance + +- For iOS/macOS selector taps with a frame, use coordinate-first resolved element activation. tvOS remains focus/remote-driven. +- Before commands to ready sessions, probe `uptime`, except for the stated startup, readiness, and recent-mutation cases. +- Keep recent-mutation health only on the runner session and only for the stated app, commands, and 5-second window. +- Invalidate dead cached runner processes without graceful `shutdown`. +- After `RUNNER_BUSY`, resend only read-only commands under their busy-specific budget. Never replay mutations. +- Before a busy runner can enter retention, `close` stops it and awaits lease release. +- After a root AX failure, clear the cached app target. Keep raw/strict errors distinct from interactive recovery payloads. + ## Context -The iOS runner is a long-lived XCTest process with an HTTP command loop. A command can appear to -complete at the daemon boundary while XCTest is already tearing down the test runner. +The iOS runner is a long-lived XCTest process with an HTTP command loop. +A command can appear complete at the daemon boundary while XCTest already stops the runner. -This was reproduced in the React Navigation playground with navigation-causing selector taps such -as `Navigate to Details` and `Back to home`. The runner resolved the button and synthesized the tap, -the app navigated, and then XCTest tried to re-resolve the original `XCUIElement`. Because the -element had disappeared, xcodebuild recorded `Failed to get matching snapshot` and ended the test -with `** TEST EXECUTE FAILED **`. The daemon had already received a successful tap response, so the -next read-only command inherited a stale cached runner. +This occurred in the React Navigation playground after selector taps that caused navigation, such as `Navigate to Details` and `Back to home`. +The runner resolved the button and synthesized the tap. +The app navigated, and XCTest then tried to resolve the original `XCUIElement` again. +Because the element had disappeared, xcodebuild recorded `Failed to get matching snapshot` and ended the test with `** TEST EXECUTE FAILED **`. +The daemon had already received a successful tap response. +Thus, the next read-only command used a stale cached runner. -Two older assumptions were wrong: +These earlier assumptions were incorrect: -- A recent successful runner response proves the runner is still healthy. -- `XCUIElement.tap()` is the safest selector-tap primitive once a selector has resolved. -- A cached `XCUIApplication` target remains safe after XCTest reports that the app's accessibility - tree cannot be serialized. +- A recent successful runner response proves continued runner health. +- After selector resolution, `XCUIElement.tap()` is the safest selector-tap primitive. +- A cached `XCUIApplication` target remains safe after XCTest cannot serialize the app's accessibility tree. ## Decision -Coordinate-first resolved element activation is the iOS/macOS selector-tap model. The runner still -uses selectors or text queries to find the semantic `XCUIElement`, but when the element has a frame, -activation taps the resolved center point instead of calling `XCUIElement.tap()`. tvOS remains -focus/remote-driven because tvOS does not support normal coordinate input. - -Ready runner sessions are probed with a short `uptime` preflight before command send. Read-only -startup commands still skip that preflight because the first successful command is the readiness -proof for a newly launched runner. Readiness probe commands skip preflight to avoid recursion. - -The daemon may additionally skip the ready-session `uptime` preflight for an explicit allowlist of -mutating interactions (`tap`, `longPress`, `drag`, `swipe`, `scroll`, `sequence`) when the same -session produced a healthy mutating response — parsed ok and not carrying `runnerFatal` — for the -same `appBundleId` within 5 seconds. This recency lives only on the `RunnerSession` object as -`lastHealthyMutation`, so it dies with every invalidation/restart, and it is recorded only after the -`runnerFatal` check, so sparse AX-fallback snapshots and `runnerFatal` payloads never refresh it. -Snapshots and other read-only responses never count as a health signal. This narrow skip is -permitted because: coordinate-first activation removed -the command-induced teardown trigger, and the lifecycle status journal plus the status-before- -invalidate recovery is the teardown-surviving status surface that resolves any ambiguous post-send -failure before invalidation. A transport failure after a skip clears the recency record and is marked -with the skip context; connection-shaped failures (refused, reset, hung up) run status recovery -instead of a blind replay, while timeout-shaped failures propagate with the skip context (the same -classification preflighted sends use). - -`uptime` is a direct runner listener probe. It is answered before command journaling, the serial -command execution queue, app activation, and main-thread XCTest dispatch. It should measure only -whether the runner is alive and accepting new HTTP requests. - -Dead cached runner processes are invalidated without graceful `shutdown`. A process that already -stopped cannot answer the shutdown request, so graceful cleanup only adds stale-listener delay. - -The runner stamps its live main-thread occupancy (`runnerMainThreadBusy`) onto every successful -response and tags the execution-watchdog timeout with the typed `MAIN_THREAD_TIMEOUT` code, so the -daemon learns the main thread is occupied from the stalling command itself, not only from a later -`RUNNER_BUSY` refusal. The daemon mirrors that occupancy on the `RunnerSession`: set on `RUNNER_BUSY` -or `MAIN_THREAD_TIMEOUT`, cleared by any other served runner reply (which reached the main thread and -therefore drained), and left intact by a transport failure or an unstamped recovered response. A -healthy `ok` is never read as proof of drain because a private-AX snapshot can be served while an -abandoned tree crawl still grinds on the XCTest main thread (#2552). - -What a `RUNNER_BUSY` refusal then costs the caller turns on the daemon's own per-command read-only -trait, not on any runner-side classification of the command: `RUNNER_COMMAND_TRAITS` in -`packages/platform-apple/src/runner/runner-command-traits.ts`, read through -`isReadOnlyRunnerCommand` where `runAppleRunnerCommand` decides whether to wrap the send in a resend -loop. The asymmetry is shipped, and asserted in `runner-client-busy-resend.test.ts`: a read-only -command treats the refusal as *not yet*, waiting the drain out on the busy-specific budget sized to -outlast it and resending until the runner answers or the window ends; a mutating command treats the -same refusal as *not mine to send again* — one attempt, and the refusal reaches the caller unwrapped -with nothing replayed. Neither path pays a status probe, because a structured reply already answered -"did my command run?" (the bypass in `runner-lifecycle.ts`). The same trait already decides the -startup-preflight skip above, so it is the single place a change to that classification lands. - -Close that would retain a runner for reuse first stops it when that occupancy is set, awaiting the -stop so the lease is released before the next request. A runner still finishing watchdog-abandoned -work does refuse every command sent to it until it drains or escalates to `RUNNER_WEDGED`, and that -fact is stated here for the close-and-pooling decision only: the read-only resend above is one live -caller waiting the same refusal out on its own budget, and `close` does not inherit that wait on the -pool's behalf. Pooling the stalled runner back hands it to the next `open` and `close` recovers -nothing. Killing the process is the only way to abort uncancellable XCTest work. This is the -`RUNNER_WEDGED` restart from #1105 applied at the close boundary rather than after the wedge -threshold elapses. - -When XCTest reports a root accessibility snapshot failure such as `kAXErrorIllegalArgument`, the -runner treats the cached app target as suspect. Interactive snapshots fail closed to a truncated -root-only payload instead of issuing more flat fallback queries against the same broken tree, and -the cached `XCUIApplication` handle is cleared so the next command reacquires the target through the -normal activation path. - -An external iOS-family Simulator relaunch or physical iOS relaunch also invalidates process-bound -target state. After replacing the app process, the daemon sends a lifecycle reset to the retained -runner so the next command reacquires `XCUIApplication`; the reset also clears process-bound snapshot -penalty and private-AX depth state. If that reset cannot be confirmed, or if relaunch fails before -reset, the daemon discards the runner session. Other physical Apple OS leaves retain their existing -runner-restart behavior until separately evidenced. - -The snapshot surface intentionally has two AX-failure shapes. Interactive fast snapshots return a -truncated success payload with `runnerFatal` so agents can still see that AX state is unavailable -and recover with a plain screenshot plus coordinate navigation. Raw or strict snapshot paths keep -returning an error because those callers requested a faithful tree, not a lossy recovery payload. +Coordinate-first resolved element activation is the iOS/macOS selector-tap model. +The runner uses selectors or text queries to find the semantic `XCUIElement`. +If the element has a frame, activation taps its resolved center point instead of calling `XCUIElement.tap()`. +tvOS remains focus/remote-driven because it does not support normal coordinate input. + +Before command send, ready runner sessions receive a short `uptime` preflight. +Read-only startup commands skip this preflight. +The first successful command proves readiness for a newly launched runner. +Readiness probe commands also skip preflight to avoid recursion. + +The daemon may also skip ready-session `uptime` for an explicit allowlist of mutating interactions: + +- `tap`. +- `longPress`. +- `drag`. +- `swipe`. +- `scroll`. +- `sequence`. + +This requires a healthy mutating response from the same session for the same `appBundleId` within 5 seconds. +The response must parse successfully and carry no `runnerFatal`. +Only the `RunnerSession` object stores this recency as `lastHealthyMutation`. +Each invalidation or restart deletes it. +The record is written only after the `runnerFatal` check. +Sparse AX-fallback snapshots and `runnerFatal` payloads therefore never renew it. +Snapshots and other read-only responses never count as health signals. + +This narrow skip is permitted because coordinate-first activation removes the command-induced teardown trigger. +The lifecycle status journal and status-before-invalidate recovery preserve status across teardown. +They resolve ambiguous post-send failures before invalidation. +After a skipped preflight, a transport failure clears recency and carries skip context. +Connection-shaped failures, such as refused, reset, or hung up, use status recovery instead of blind replay. +Timeout-shaped failures propagate with skip context. +This is the same classification used by preflighted sends. + +`uptime` directly probes the runner listener. +The listener answers before command journaling, the serial execution queue, app activation, and main-thread XCTest dispatch. +It should measure only whether the runner is alive and accepts new HTTP requests. + +Invalidate dead cached runner processes without graceful `shutdown`. +A stopped process cannot answer a shutdown request. +Graceful cleanup would only add stale-listener delay. + +Every successful runner response carries live main-thread occupancy as `runnerMainThreadBusy`. +An execution-watchdog timeout carries typed code `MAIN_THREAD_TIMEOUT`. +Thus, the stalled command itself informs the daemon of occupancy before a later `RUNNER_BUSY` refusal. +The daemon mirrors occupancy on `RunnerSession`: + +- Set it on `RUNNER_BUSY` or `MAIN_THREAD_TIMEOUT`. +- Clear it on any other served runner reply, which reached the main thread and therefore drained. +- Preserve it after a transport failure or unstamped recovered response. + +A healthy `ok` never proves drain. +A private-AX snapshot can succeed while an abandoned tree crawl continues on the XCTest main thread (#2552). + +The daemon's per-command read-only trait determines behavior after `RUNNER_BUSY`. +No runner-side command classification controls it. +`RUNNER_COMMAND_TRAITS` in `packages/platform-apple/src/runner/runner-command-traits.ts` owns this trait. +`runAppleRunnerCommand` reads it through `isReadOnlyRunnerCommand` when it selects the resend loop. +`runner-client-busy-resend.test.ts` asserts these distinct behaviors: + +- A read-only command waits for drain under a busy-specific budget sized to outlast it. + It resends until the runner answers or the window ends. +- A mutating command makes one attempt. + The refusal reaches the caller without a wrapper or replay. + +Neither path performs a status probe because the structured reply already states whether the command ran. +`runner-lifecycle.ts` implements this bypass. +The same trait controls the startup-preflight skip above. +It is therefore the single owner for classification changes. + +If `close` would retain an occupied runner for reuse, it first stops that runner. +It awaits the stop so the lease is released before the next request. +A runner with watchdog-abandoned work refuses every command until drain or escalation to `RUNNER_WEDGED`. +This fact governs close and retention only. +The read-only resend loop is one live caller that waits under its own budget. +`close` does not inherit that wait for the pool. +Returning the stalled runner to the pool gives it to the next `open` without recovery. +Only process termination can abort uncancellable XCTest work. +This applies the `RUNNER_WEDGED` restart from #1105 at close instead of after the wedge threshold. + +If XCTest reports a root accessibility snapshot failure such as `kAXErrorIllegalArgument`, the runner treats the cached app target as suspect. +Interactive snapshots fail closed with a truncated root-only payload. +They do not issue more flat fallback queries against the broken tree. +The runner clears the cached `XCUIApplication` handle. +The next command reacquires the target through normal activation. + +An external iOS-family Simulator relaunch or physical iOS relaunch also invalidates process-bound target state. +After app process replacement, the daemon sends a lifecycle reset to the retained runner. +The next command then reacquires `XCUIApplication`. +The reset also clears process-bound snapshot penalty and private-AX depth state. +If the reset cannot be confirmed, the daemon discards the runner session. +If relaunch fails before reset, it also discards the session. +Other physical Apple OS leaves retain existing runner-restart behavior until separate evidence supports a change. + +The snapshot surface intentionally has two AX-failure forms. +Interactive fast snapshots return truncated success with `runnerFatal`. +Agents can see AX unavailability and recover through a plain screenshot and coordinate navigation. +Raw or strict snapshot paths retain errors because their callers request a faithful tree instead of lossy recovery. ## Consequences -Navigation-causing selector taps no longer couple command success to XCTest's post-tap element -bookkeeping. If the target disappears because navigation happened, the tap remains a normal -successful interaction and the runner should stay alive. - -If xcodebuild still exits for another reason, the next command detects the stale runner through -process/liveness checks and avoids the old 15-second graceful-shutdown wait. The remaining latency is -fresh xcodebuild runner startup, not a stale transport stall. - -A `close` then `open` on a runner still draining abandoned main-thread work now pays a fresh runner -boot instead of inheriting the stalled one, so the wedge no longer survives the close/open cycle. -Runners that are ready and have no unanswered exchanges can remain warm when retention is requested; -iOS-family simulators additionally require an observed live listener. Close retires unfinished -startup and its recovery authority. The retained generation owns its idle timer and listener -observation; successful reuse ends retention, listener loss or idle expiry disposes that generation, -and graceful daemon shutdown stops idle retained generations before exiting. Active ready -generations retain the existing handoff policy. - -The daemon no longer models a generic "recent success" cache as a runner-health signal. A proven -healthy mutating response for the same app — recorded only after the `runnerFatal` check and only -for allowlisted interactions — is now a real end-to-end liveness proof (HTTP listener through to the -app target), so a hot loop of allowlisted interactions skips the per-command `uptime` request while -still re-earning each skip from another healthy mutation. The earlier unconditional `uptime` before -every ready-session command remains the default for non-allowlisted commands and after any -invalidation, stale record, app-bundle change, or absent record. - -Apps with broken accessibility trees may still be impossible for XCTest to inspect deeply, but one -failed snapshot no longer teaches the runner to keep using a suspect cached app target or to amplify -the failure by walking every interactive element query. - -An `XCTEST_RECORDED_FAILURE` returned after a tap is treated as an ambiguous outcome at the daemon -boundary. When a usable retained snapshot exists, the daemon takes one same-presentation post-action -capture. A changed accessibility digest is reported as success with an explicit warning so agents do -not blindly repeat a tap that may already have navigated; unchanged, sparse, mismatched, or unavailable -evidence remains the original failure. - -iOS-family Simulator and physical iOS relaunch keep the healthy XCTest process warm without carrying -an app target across process identity. The reset adds one local runner request instead of paying for a -runner restart and clears the old process's hostile-screen capture penalty before the replacement is -reacquired. - +Selector taps that cause navigation no longer depend on XCTest's post-tap element resolution. +If navigation removes the target, the tap remains a normal successful interaction. +The runner should remain alive. + +If xcodebuild exits for another reason, process/liveness checks detect the stale runner on the next command. +They avoid the old 15-second graceful-shutdown wait. +Remaining latency comes from fresh xcodebuild runner startup instead of stale transport delay. + +A `close` then `open` during abandoned main-thread work starts a fresh runner instead of reusing the stalled runner. +The wedge therefore does not persist across close/open. +When retention is requested, ready runners without unanswered exchanges can remain active for reuse. +iOS-family simulators also require an observed live listener. +Close retires unfinished startup and its recovery authority. +The retained generation owns its idle timer and listener observation. +Successful reuse ends retention. +Listener loss or idle expiry disposes that generation. +Graceful daemon shutdown stops idle retained generations before exit. +Active ready generations retain the existing handoff policy. + +The daemon does not treat generic recent success as runner health. +A proven healthy mutating response for the same app provides end-to-end liveness evidence from HTTP listener to app target. +It counts only after the `runnerFatal` check and only for allowlisted interactions. +A repeated series of allowlisted interactions can therefore skip per-command `uptime`. +Each healthy mutation establishes permission for the next skip. +Unconditional `uptime` remains the default for non-allowlisted commands and after: + +- Invalidation. +- A stale record. +- An app-bundle change. +- An absent record. + +XCTest may still be unable to examine deeply an app with a broken accessibility tree. +However, one failed snapshot no longer causes continued use of a suspect target or additional failures from every interactive element query. + +After a tap, `XCTEST_RECORDED_FAILURE` is an ambiguous outcome at the daemon boundary. +If a usable retained snapshot exists, the daemon takes one same-presentation post-action capture. +A changed accessibility digest returns success with an explicit warning. +This prevents blind repetition of a tap that may already have navigated. +Unchanged, sparse, mismatched, or unavailable evidence retains the original failure. + +iOS-family Simulator and physical iOS relaunch retain a healthy XCTest process without retaining an app target across process identity. +The reset adds one local runner request instead of a runner restart. +Before replacement target acquisition, it clears the old process's hostile-screen capture penalty. From c10a428b941effba0c9175f006e99a9dbdab7667 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:58:09 +0200 Subject: [PATCH 09/55] docs(adr): write ADR 0006 in ASD-STE100 --- docs/adr/0006-daemon-rpc-protocol-version.md | 189 +++++++++++-------- 1 file changed, 107 insertions(+), 82 deletions(-) diff --git a/docs/adr/0006-daemon-rpc-protocol-version.md b/docs/adr/0006-daemon-rpc-protocol-version.md index c286210963..8cca391e8c 100644 --- a/docs/adr/0006-daemon-rpc-protocol-version.md +++ b/docs/adr/0006-daemon-rpc-protocol-version.md @@ -4,115 +4,140 @@ Accepted +## Rules at a glance + +- Before remote JSON-RPC requests, compare the daemon's `rpcProtocolVersion` from `/health`. +- Package `version` is diagnostic only and must not control compatibility. +- Permit missing `rpcProtocolVersion` unless a later security or protocol decision explicitly retires legacy compatibility. +- Cache probes only with complete endpoint and upstream instance IDs. Send conditional headers on every cached RPC. +- After a stale-instance refusal, probe again. Retry once only if the new peer's protocol is compatible. +- Change `rpcProtocolVersion` only for unsafe compatibility changes to existing HTTP RPC contracts. +- Keep additive changes at the existing protocol version. +- Treat an absent `capabilities` entry in `daemon.json` as "cannot do this". Never infer a capability from the package version. +- A client never signals a daemon that did not accept a quiesce, except an unreachable daemon it retires deliberately. +- Force-retire a daemon at the drain deadline only when it fails its liveness probes and its command line proves it is a daemon process. + ## Context -`agent-device` can run a client on one machine and reach a daemon on another machine through the -HTTP JSON-RPC transport, including `agent-device proxy` in front of a local macOS daemon. Client and -remote host package versions can differ. Many version skews are safe because the client sends a -stable command request envelope and the daemon executes only the requested command. +`agent-device` can run a client on one machine and connect to a daemon on another through HTTP JSON-RPC. +This includes `agent-device proxy` in front of a local macOS daemon. +Client and remote host package versions can differ. +Many version differences are safe because the client sends a stable command request envelope and the daemon executes only the requested command. -Package semver is too strict for this boundary: a 0.15 client and 0.17 remote daemon can often -interoperate. At the same time, the client needs a cheap way to fail before sending command RPC when -the transport protocol itself is no longer compatible. +Package semver is too strict for this boundary. +A 0.15 client and 0.17 remote daemon can often interoperate. +However, the client needs a low-cost compatibility check before command RPC when the transport protocol is incompatible. ## Decision -The daemon HTTP `/health` payload advertises a `rpcProtocolVersion` integer alongside service and -package version metadata. Remote clients compare this value before sending JSON-RPC requests. - -Package `version` is diagnostic only and must not be used as a compatibility gate. Missing -`rpcProtocolVersion` is treated as a legacy remote daemon and is allowed unless a later security or -protocol decision explicitly retires legacy compatibility. - -A persistent client may cache a successful probe when `/health` advertises an instance ID for -both the daemon endpoint and, if proxied, its upstream daemon. It sends those IDs as conditional -headers on every cached RPC. A daemon refuses a stale instance with a typed HTTP 409 response -before dispatch, and a proxy refuses its own stale instance before forwarding. The proxy passes -the expected upstream instance to the daemon, which makes the same pre-dispatch check. On a -mismatch the client clears the cache, probes `/health` again, and retries the refused RPC once -only when the new peer's protocol is compatible. Peers without complete instance IDs retain -per-command probes. A peer that advertises an instance ID must continue honoring the conditional -RPC headers in future versions: older clients may hold a cached identity across a restart, and -must receive a pre-dispatch refusal before retrying a mutating command. - -`rpcProtocolVersion` changes only when an older client and newer daemon, or newer client and older -daemon, cannot safely communicate over the HTTP RPC boundary for existing commands. Bump it for -breaking changes to: +The daemon HTTP `/health` payload advertises integer `rpcProtocolVersion` beside service and package version metadata. +Before JSON-RPC requests, remote clients compare this value. + +Package `version` is diagnostic only and must not control compatibility. +Missing `rpcProtocolVersion` identifies a legacy remote daemon. +It is permitted unless a later security or protocol decision explicitly retires legacy compatibility. + +A persistent client may cache a successful probe if `/health` advertises instance IDs for the daemon endpoint and any proxied upstream daemon. +On every cached RPC, it sends those IDs as conditional headers. +Before dispatch, a daemon refuses a stale instance with a typed HTTP 409 response. +Before it forwards a request, a proxy refuses its own stale instance. +The proxy passes the expected upstream instance to the daemon, which performs the same pre-dispatch check. +After a mismatch, the client clears the cache and probes `/health` again. +Only if the new peer's protocol is compatible does it retry the refused RPC once. +Peers without complete instance IDs retain per-command probes. +A peer that advertises an instance ID must continue to honor conditional RPC headers in future versions. +Older clients may retain a cached identity across a restart. +Before retrying a mutating command, they must receive a pre-dispatch refusal. + +`rpcProtocolVersion` changes only when an older client and newer daemon, or newer client and older daemon, cannot safely communicate for existing commands. +This applies to the HTTP RPC boundary. +Increment it for incompatible changes to: - HTTP route requirements for `/health`, `/rpc`, `/upload`, or `/artifacts/*`. - Authentication semantics required to authorize RPC, upload, or artifact requests. -- JSON-RPC envelope shape, method naming, request id handling, or command request projection. -- Response, error, artifact, upload, or progress-stream framing that existing clients parse. -- Existing command request or response contracts when the old side would misinterpret the payload - rather than fail clearly. +- JSON-RPC envelope shape, method names, request id handling, or command request projection. +- Response, error, artifact, upload, or progress-stream framing parsed by existing clients. +- Existing command request or response contracts that the older side would misinterpret instead of clearly rejecting. -Do not bump it for additive changes: +Do not increment it for additive changes: - New commands. -- New optional request fields or flags that older daemons can ignore or reject with a normal - command error. +- New optional request fields or flags that older daemons can ignore or reject with a normal command error. - New optional response fields that older clients can ignore. -- Package version changes, refactors, or implementation-only daemon/proxy changes. +- Package version changes, refactors, or daemon/proxy changes limited to implementation. -When a single command needs finer-grained compatibility in the future, prefer command-level feature -or capability metadata over bumping the whole RPC protocol, unless the shared transport contract is -also affected. +Unless the shared transport contract also changes, prefer command-level metadata for finer command compatibility. +Feature or capability metadata can supply this information. ## Alternatives Considered -- Gate by package version: simple, but rejects compatible version skew and makes proxy usage brittle - across frequent releases. -- Do not check compatibility: maximizes compatibility but fails later, after the client has sent an - RPC payload that a remote daemon may parse incorrectly. -- Full schema negotiation: more precise, but too much machinery for the current JSON-RPC boundary. +- **Use package version for compatibility.** This is simple but rejects compatible version differences. + Frequent releases also make proxy use unreliable across versions. +- **Do not check compatibility.** This maximizes attempted compatibility. + However, failure occurs after the client sends an RPC payload that a remote daemon may parse incorrectly. +- **Negotiate full schemas.** This is more precise but requires excessive machinery for the current JSON-RPC boundary. ## Consequences -Protocol-breaking changes must update `DAEMON_RPC_PROTOCOL_VERSION`, tests that assert `/health` -metadata, and at least one remote-client regression test that proves mismatched protocols fail before -command RPC. +Protocol-breaking changes must update: -The bump rules above are enforced rather than remembered (#1432). `test/wire-compat/surface.ts` -declares the wire surface grouped by the bullets in this decision, and `test/wire-compat/ledger.json` -records what each declaration hashes to and at which protocol version. The unit lane fails when a -declaration's shape moves without the ledger following, and `pnpm check:daemon-wire-compat` compares -the ledger against the last released tag: drift since that release must carry either a bump or a -`compatibleChanges` acknowledgment invoking the additive list above. Where a bullet is only partly -digestible — the `/health` and `/rpc` path literals inside `http-server.ts` — the manifest records -the gap and its reason instead of implying coverage. See `test/wire-compat/README.md`. +- `DAEMON_RPC_PROTOCOL_VERSION`. +- Tests for `/health` metadata. +- At least one remote-client regression test that proves mismatched protocols fail before command RPC. -Legacy remote daemons without `rpcProtocolVersion` remain reachable. This keeps the first release of -the proxy compatible with older HTTP daemons, but it means absence of the marker is not proof of -compatibility. +Checks enforce the version rules above (#1432). +`test/wire-compat/surface.ts` declares the wire surface, grouped by the decision's categories. +`test/wire-compat/ledger.json` records each declaration's hash and protocol version. +The unit lane fails if a declaration's shape changes without a corresponding ledger change. +`pnpm check:daemon-wire-compat` compares the ledger with the last released tag. +Changes since that release must have either a protocol increment or a `compatibleChanges` acknowledgment under the additive list above. +Some categories are only partly represented in digests, including `/health` and `/rpc` path literals inside `http-server.ts`. +For those categories, the manifest records the gap and its reason instead of implying coverage. +See `test/wire-compat/README.md`. + +Legacy remote daemons without `rpcProtocolVersion` remain reachable. +This keeps the first proxy release compatible with older HTTP daemons. +However, an absent marker does not prove compatibility. ## Daemon capabilities and quiesce -A daemon on the local state directory publishes `capabilities` in `daemon.json`. This is the -command-level metadata the decision above prefers to a protocol bump. A daemon that predates a -capability omits it, so a client treats absence as "cannot do this" and never infers it from the -package version. +A daemon on the local state directory publishes `capabilities` in `daemon.json`. +This is the command-level metadata that the decision above prefers over a protocol version bump. +A daemon that predates a capability omits it. +A client treats an absent capability as "cannot do this". +A client never infers a capability from the package version. `quiesce` is how a client replaces an incompatible daemon without destroying other clients' work. The client sends `daemon_quiesce` with the daemon token and the pid and start time recorded in -`daemon.json`. The daemon checks the token and that it is that instance, refuses a request that -reached it as a remote-origin one, then closes admission and counts what it holds in a single -synchronous step: requests in flight, sessions, repair sessions and live leases, and whether the -daemon is still starting up (`startingUp`), because before that it has no drain path. Any of them -refuses with `daemon-busy` and reopens admission in that same step. Otherwise it answers -`draining`, keeps admission closed, and shuts down. The request is answered before dispatch, so -it joins neither the in-flight count nor the command registry. +`daemon.json`. +The daemon checks the token and checks that it is that instance. +The daemon refuses a request that reached it as a remote-origin request. +Then the daemon closes admission and counts what it holds in one synchronous step: + +- Requests in flight. +- Sessions. +- Repair sessions and live leases. +- Whether the daemon is still starting up (`startingUp`). Before startup completes, the daemon has no drain path. + +If any of them is present, the daemon refuses with `daemon-busy` and reopens admission in that same step. +Otherwise, the daemon answers `draining`, keeps admission closed, and shuts down. +The daemon answers the request before dispatch, so it joins neither the in-flight count nor the command registry. The daemon accepts a quiesce only when it can prove its own start time and the request names both -the pid and that start time; a daemon that cannot is never addressed. - -After `draining` the client signals nothing and waits, up to a bounded deadline, for the daemon to -exit by itself and release its registration: the daemon runs its own shutdown, which a signal would -cut short. Only a daemon still running at the deadline that also fails its liveness probes is -force-retired, and only when its command line proves it is a daemon process. A daemon that still -answers is left alone and the call fails with `daemon-drain-timeout`. A client never signals a -daemon that did not accept a quiesce, except an unreachable one it retires deliberately. - -The census counts what the contract names and nothing else. A resumable upload still streaming and -a host-administration human-control hold that names no lease are outside it, the same boundary -idle reap draws, so a quiesce can end them. A daemon started with an HTTP auth hook marks every -HTTP request remote-origin and so refuses a quiesce over HTTP; the socket transport is unaffected. +the pid and that start time. A daemon that cannot prove this is never addressed. + +After `draining`, the client sends no signal. The client waits, up to a bounded deadline, for the +daemon to exit by itself and release its registration. The daemon runs its own shutdown. A signal +would cut that shutdown short. +A daemon is force-retired only when it is still running at the deadline, fails its liveness probes, +and its command line proves it is a daemon process. +A daemon that still answers is left alone. The call fails with `daemon-drain-timeout`. +A client never signals a daemon that did not accept a quiesce. The exception is an unreachable +daemon that the client retires deliberately. + +The census counts what the contract names and nothing else. +A resumable upload that is still streaming is outside the census. A host-administration +human-control hold that names no lease is also outside it. This is the same boundary that the idle +reap draws. So a quiesce can end both. +A daemon started with an HTTP auth hook marks every HTTP request as remote-origin. Such a daemon +refuses a quiesce over HTTP. The socket transport is unaffected. From 5501a0272b0a8e67bf8206d527be4da248729330 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:59:15 +0200 Subject: [PATCH 10/55] docs(adr): write ADR 0007 in ASD-STE100 --- docs/adr/0007-remote-device-leases.md | 446 +++++++++++++++----------- 1 file changed, 256 insertions(+), 190 deletions(-) diff --git a/docs/adr/0007-remote-device-leases.md b/docs/adr/0007-remote-device-leases.md index b69a1f181e..9a4100a027 100644 --- a/docs/adr/0007-remote-device-leases.md +++ b/docs/adr/0007-remote-device-leases.md @@ -4,218 +4,284 @@ Accepted +## Rules at a glance + +- Allocate device leases lazily after device, backend, and provider resolution. +- Keep lease mutation and request execution under the same daemon request lock. +- Generated connection profiles must strip daemon and Metro bearer tokens. +- A session cannot outlive its lease. Teardown clears its device claim even if the device does not answer. +- Report request-ended leases once on the outermost response. Clients must not release them again. +- Wanted admitted work preserves its lease. Canceled work neither defers expiry nor renews it. +- Before admission, protect long client phases with lease heartbeats under the lease's existing window. +- Human-control holds fence mutations through registry-owned admission and drain behavior. +- Only host administrators allocate and release `macos-app` leases. Confine each admitted request to the leased app. + ## Context -Remote daemon users need a clear ownership boundary before commands reach a -platform runner or helper. Shared proxy and hosted providers need ownership to -include the selected device and connection provider, not only tenant/run. +Remote daemon users need a clear ownership boundary before commands reach a platform runner or helper. +Shared proxies and hosted providers need ownership to include the selected device and connection provider, in addition to tenant/run. -Runner and helper processes already have backend-specific mutual exclusion. That -guard protects platform tooling, not remote client ownership, so surfacing those -errors directly makes device contention harder to recover from. +Runner and helper processes already have backend-specific mutual exclusion. +This guard protects platform tools, not remote client ownership. +Direct exposure of those errors makes device contention harder to resolve. ## Decision -A remote device lease is logical ownership of one selected device by one -remote client for a connection provider such as `proxy`, cloud, or `limrun`. - -`connect` establishes connection profile and client identity. Lease allocation -is lazy and happens when a device, backend, and provider are known. - -A runner/process lease is a backend helper guard and is not a user/client -ownership boundary. It stays below daemon device leases and should not be -weakened or replaced by them. - -`open` is the natural point to acquire a device lease because target resolution -and session creation meet there. Commands after `open` must refresh the lease; -no activity for five minutes should make the device available again. - -Lease admission, heartbeat, stored session lease refresh, and request execution -must run under the same daemon request lock. Scope resolution may happen before -the lock, but lease ownership mutation must not. - -Generated connection profiles are non-secret. They may persist routing and -lease metadata, but must strip daemon and Metro bearer tokens. Tokens are -supplied in-memory for the current command or through environment/CLI token -paths. - -The proxy process is expected to be long-lived and self-serve. Recovery from a -stale or expired device lease should not require restarting the proxy. - -A caller that owns a lease's lifetime, allocating it and releasing it itself, -can allocate with `retainOnClose`. Session `close` then leaves that lease and -its provider device in place, and only `leases.release`, expiry, or daemon -shutdown ends it. The default is unchanged so the CLI's proxy sharing still -frees devices on `close`. The allocated lease reports `retainOnClose: true` -only when the daemon honored it, and only the lease's own client can turn it -on for a lease the run already holds. An unexpired `retainOnClose` lease keeps -an idle daemon alive; other leases, including human-control holds, do not. A -retained lease the caller stops heartbeating expires after its `ttlMs`, and idle -reap then shuts the daemon down within one more idle window. - -A session cannot outlive the lease it was opened under. When the lease is -released (tenant `lease_release` or host `DELETE /admin/leases/`) or -expires, the daemon tears the session down under its execution locks and clears -its device claim, even when the device no longer answers. The daemon sweeps for -expired leases every second, so expiry does not wait for a later request; -in-flight work and human-control holds still keep a lease from expiring. Once -daemon shutdown begins, no new lease-end teardown starts, and shutdown waits for -any that already started before it tears sessions down itself. - -The daemon ends a session's lease when a request closes that session, unless the lease was -allocated with `retainOnClose`. The reply to that request reports each lease it ended once, at -response level, as `endedLeases: [{ leaseId, outcome: 'released' }]`: in `data` on success, in -`error.details` on failure, including a close that failed after the release. This holds for every -request that can close a session: `close`, a `batch` close step, a `replay` or `test` script's -`close`, and the session cleanup `test` runs after each script. Nested requests record on the -outermost request's ledger, so a step reply never repeats it. A close that finds its lease already -inactive reports it as ended. A client never sends a release after such a request. A client that -stores the lease id forgets it when the reply names it, and a gateway that relays the reply reads -the lease end from it. Close and lease-end teardown take the same execution locks, so whichever -runs second finds the session gone: a close after a lease end has no session and ends no lease, and -a client that still stores that lease id finds it inactive on its next heartbeat and allocates a -new lease under a fresh id. +A remote device lease is logical ownership of one selected device by one remote client for a connection provider. +Examples include `proxy`, cloud, and `limrun`. + +`connect` establishes the connection profile and client identity. +Lease allocation is lazy. +It occurs after device, backend, and provider resolution. + +A runner/process lease guards a backend helper. +It does not define user/client ownership. +It remains below daemon device leases and should not be weakened or replaced by them. + +`open` is the natural place to acquire a device lease because target resolution and session creation meet there. +After `open`, commands must refresh the lease. +After five minutes without activity, the device should become available again. + +These operations must use the same daemon request lock: + +- Lease admission. +- Heartbeat. +- Stored session lease refresh. +- Request execution. + +Scope resolution may occur before the lock. +Lease ownership mutation must not. + +Generated connection profiles are non-secret. +They may persist route and lease metadata, but must strip daemon and Metro bearer tokens. +Tokens are supplied in memory for the current command or through environment/CLI token paths. + +The proxy process is expected to remain active for self-service use. +Recovery from stale or expired device leases should not require a proxy restart. + +A caller that allocates and releases a lease itself can allocate with `retainOnClose`. +Session `close` then retains that lease and its provider device. +Only `leases.release`, expiry, or daemon shutdown ends it. +The default remains unchanged, so CLI proxy sharing still frees devices on `close`. +The allocated lease reports `retainOnClose: true` only if the daemon honored it. +Only the lease's own client can enable it for a lease already held by the run. +An unexpired `retainOnClose` lease keeps an idle daemon alive. +Other leases, including human-control holds, do not. +If the caller stops heartbeats, a retained lease expires after its `ttlMs`. +Idle reap then stops the daemon within one more idle window. + +A session cannot outlive the lease under which it opened. +On release or expiry, the daemon tears the session down under its execution locks and clears its device claim. +This applies even when the device no longer answers. +Release can use tenant `lease_release` or host `DELETE /admin/leases/`. +The daemon examines expired leases every second. +Thus, expiry does not require a later request. +In-flight work and human-control holds still prevent lease expiry. +After daemon shutdown starts, no new lease-end teardown starts. +Shutdown waits for already-started teardown before it tears sessions down itself. + +When a request closes a session, the daemon ends its lease unless it was allocated with `retainOnClose`. +The response reports each lease ended by that request once, at response level: +`endedLeases: [{ leaseId, outcome: 'released' }]`. +Success reports it in `data`. +Failure reports it in `error.details`, including when close fails after release. +This applies to every request that can close a session: + +- `close`. +- A `batch` close step. +- `close` in a `replay` or `test` script. +- Session cleanup after each `test` script. + +Nested requests record on the outermost request's ledger. +Step replies never repeat that record. +If close finds its lease already inactive, it reports the lease as ended. +A client never sends a release after such a request. +If a client stores the lease id, it forgets that id when the reply names it. +A gateway that relays the reply reads the lease end from it. + +Close and lease-end teardown use the same execution locks. +Whichever runs second finds the session gone. +Thus, close after lease end has no session and ends no lease. +If a client still stores the id, its next heartbeat finds an inactive lease. +It allocates a new lease with a fresh id. ## Consequences -Device contention can fail before platform execution with an explicit -device-lease error that includes the backend, provider, selected device key, and -owning lease expiry. +Device contention can fail before platform execution with an explicit device-lease error. +The error includes backend, provider, selected device key, and owning lease expiry. -Backend-only leases remain valid for older remote clients, while provider-aware -clients get device-level contention and clearer recovery. +Backend-only leases remain valid for older remote clients. +Provider-aware clients receive device-level contention and clearer recovery. ## Admitted request work -A lease renews when a request is admitted and never again while that request runs, so an admitted -command slower than the inactivity TTL used to expire the lease that was paying for its own device -and tear the session down underneath the client still waiting for its result. Admitted work -therefore preserves its lease the way a human-control hold does: while the request is still wanted -it defers expiry, and finishing while still wanted renews the lease for its existing inactivity TTL -from the moment the work ended. A request whose client hung up preserves nothing — it neither defers -expiry past that cancellation nor renews the lease when it finally lands — so a handler that ignores -its cancellation cannot hold a rented device open. - -Which leases this reaches depends on the inactivity TTL the client asked for: the daemon default is -one minute, while a cloud WebDriver connection profile asks for ten. A single command that runs -longer than its own lease is therefore ordinary on the default and only reachable through a profile -on the longer one. +Admission renews a lease once. +The request does not renew it again during execution. +Without preservation, a command longer than its inactivity TTL could expire its own device lease and lose its session before response. +Admitted work therefore preserves its lease like a human-control hold. +While the request remains wanted, it defers expiry. +If work finishes while still wanted, it renews the existing inactivity TTL from completion time. +After client disconnect, the request preserves nothing. +It neither defers expiry beyond cancellation nor renews the lease when work eventually finishes. +Thus, a handler that ignores cancellation cannot retain a rented device. + +The client's requested inactivity TTL determines which commands need preservation. +The daemon default is one minute. +A cloud WebDriver connection profile requests ten minutes. +A single command longer than its lease is therefore ordinary with the default. +The longer window requires a profile. ## Provider allocation -Allocation is admitted work too. The registry records a lease before a hosted provider creates the -session behind it, and creating that session can take longer than the lease's inactivity TTL, so a -lease timed from its record was already expired, or nearly so, when its client first received it, and -the paid session it pointed at was orphaned. The allocation therefore holds a work pass while the -provider allocates: the lease cannot expire underneath it, and a successful allocation still wanted by -its requester starts the inactivity TTL from the moment allocation completed. A requester that hung up -preserves nothing, and its allocation is released as before. +Allocation is also admitted work. +The registry records a lease before a hosted provider creates its session. +Session creation can exceed the lease's inactivity TTL. +Without preservation, the client could receive an expired or nearly expired lease and leave a paid session orphaned. +Allocation therefore holds a work pass while the provider allocates. +The lease cannot expire during that work. +If successful allocation remains wanted, the inactivity TTL starts at completion. +A disconnected requester preserves nothing, and its allocation is released as before. ## Client-side work that precedes admission -Protecting admitted work covers nothing that happens before a request is admitted. Installing an -artifact uploads it from the caller while the request that will consume it has not been admitted yet, -so an upload slower than the lease's inactivity window expired the lease paying for the device the -bytes were going to (#2946). The caller therefore beats the lease over the ordinary transport for as -long as that phase runs, naming the lease scope exactly as the command named it and no payload of its -own. An install names no window, so its beats renew for the window the lease already carries; a -caller that did name one keeps renewing on it. Resolving an absent window to the registry default -instead — which is what a heartbeat used to do — quietly shortened every lease allocated above that -default, which is the other half of why the upload could not survive. Request admission had its own -copy of that mistake: it named a proxy-specific default for every admitted request, so a lease -allocated longer than the default lost its window on the next command. Admission renews on the lease's -window too; the window a lease carries is the one its client named when it allocated. - -The first beat is fired when the phase starts, not one cadence in, because a beat is what proves the -lease the upload is spending time on is still alive — a lease shorter than any fixed cadence would -otherwise lapse before anything renewed it. A lease already gone is caught early, but not before the -phase begins: hashing, the preflight, and the start of the stream can all run while the first beat is -still outstanding, so what the first beat buys is that the loss is learned during the upload rather -than after it. Each beat answers with the window it just renewed, and the loop arms its successor when -the beat starts rather than when it settles, so a beat that never answers is abandoned on schedule -instead of taking the schedule with it. A phase that settles does not wait on a beat still in flight. - -Cadence and budget are deliberately two numbers. A beat is armed every third of the window, but it is -allowed the whole window to answer in. Budgeting a beat at its cadence instead would give a heartbeat -that needs more than a third of a window for its round trip — a tunneled proxy, an uplink the upload -itself is saturating — no way to answer at all, and every beat would re-time-out on exactly the slow -links the beat exists to protect. The window, not the cadence, is the interval in which being alive -still matters, so it is also the longest a beat's answer is worth waiting for. - -That ceiling is the shorter of the window and the heartbeat's own request policy, currently ninety -seconds. A ten-minute lease therefore gives a beat ninety seconds rather than the full window, which -costs nothing the beat exists to buy: the daemon renews the lease when it handles the request, before -it answers, so a round trip abandoned at the cap has still extended the lease if the request ever -landed — and if it never landed, a transport that has stopped carrying traffic is not served by holding -the socket open for the rest of a window measured in minutes. What the cap refuses is exactly that -idle hold. It binds only on windows longer than the policy, and on those the beat already has the -longest answer deadline any lease gets; shorter windows are bounded by their own window, as above. - -A beat is a fresh request each time, never the protected request rewritten: a request identity is -what a timed-out beat is canceled under, and beats must not inherit each other's cancellation. A beat -that finds the lease gone, or finds that this request can never renew it — its scope is missing or -belongs to another lease, or the daemon rejects the scope and window the beat itself was built with, -which no successor will ask differently — ends the phase with that error and cancels the upload rather -than finishing bytes against a device nobody owns or a lease that will stop renewing. A beat that fails -for any other reason is reported and survived, because a later beat covers one lost request — including -one that was abandoned at its budget rather than answered, which is what makes that promise hold for a -stalled round trip and not only for one that fails fast. +Admitted-work preservation does not cover earlier client work. +Artifact install uploads bytes before admission of the request that consumes them. +Without client protection, an upload longer than the inactivity window could expire the device lease (#2946). +The caller therefore sends lease heartbeats over the ordinary transport throughout that phase. +It names exactly the command's lease scope and supplies no payload of its own. +An install specifies no window, so its heartbeats renew the lease's existing window. +If a caller specifies a window, it continues to renew that window. + +Resolving an absent window to the registry default would shorten every lease allocated above that default. +Admission previously had the same defect through a proxy-specific default on each request. +Admission therefore also renews the existing lease window. +The lease retains the window named by its client at allocation. + +The first heartbeat starts with the phase, without a cadence delay. +It proves that the lease used by the upload remains alive. +Otherwise, a lease shorter than a fixed cadence could expire before renewal. +An already-lost lease is detected early, but not before phase start. +Hashing, preflight, and initial stream work can run while the first heartbeat remains outstanding. +Thus, loss becomes known during upload instead of afterward. +Each heartbeat reports the window it renewed. +The loop schedules its successor when the heartbeat starts, not when it completes. +A heartbeat without an answer is therefore abandoned on schedule without stopping the schedule. +A completed phase does not wait for an in-flight heartbeat. + +Cadence and budget intentionally differ. +A heartbeat starts every third of the window but may use the whole window for its answer. +A cadence-sized budget would always reject round trips longer than one third of the window. +This includes tunneled proxies and uplinks saturated by the upload. +Those slow links are exactly where heartbeat protection is needed. +The window defines how long continued liveness matters. +It therefore also defines the maximum useful wait for an answer. + +The actual ceiling is the shorter of the window and the heartbeat's request policy, currently ninety seconds. +Thus, a ten-minute lease permits a ninety-second answer budget. +The daemon renews the lease when it handles the request, before its answer. +A round trip abandoned at the cap has still extended the lease if the request reached the daemon. +If it never arrived, retaining a socket for the rest of a multi-minute window does not restore transport. +The cap refuses that idle wait. +It applies only to windows longer than the policy. +Those heartbeats already receive the longest answer deadline available to a lease. +Shorter windows use their own window as the bound. + +Each heartbeat is a fresh request, never a rewritten protected request. +Request identity controls cancellation of a timed-out heartbeat. +Heartbeats must not inherit each other's cancellation. +A heartbeat ends the phase with its error and cancels upload if it finds: + +- A lost lease. +- A missing scope. +- A scope owned by another lease. +- A daemon refusal of the heartbeat's own scope and window that an identical successor cannot correct. + +This prevents completed uploads for unowned devices or leases that cannot renew. +Any other heartbeat failure is reported without stopping the phase. +A later heartbeat can recover from one lost request. +This includes a request abandoned at its budget, so recovery applies to stalled round trips as well as immediate failures. ## Human control -Human-control holds coexist with an open remote session. They belong to `LeaseRegistry` and use -the same backend/provider/device contention key as device-aware leases. Hold heartbeat, expiry, -lease preservation, and release refresh are one registry-owned lifecycle. Releasing or expiring the -last hold gives the lease its existing inactivity TTL again; expiry uses the hold's expiry instant. - -Tenant hold operations are ordinary daemon RPCs admitted through `request-admission.ts`. Their -device comes only from the admitted lease. Host administration is a distinct loopback capability -authenticated with the daemon token, never a tenant credential; tenants cannot modify host holds. - -Mutation admission derives from existing recording effects, observation-class inventory, and -observability semantics. Takeover and lease heartbeats are exempt from the mutation fence, not from -their ownership checks. Unknown effects are treated as mutations. A pending activation fences new -mutations and drains those already admitted before reporting active; advisory execution locks alone -do not establish this guarantee for fresh sessions. - -Activation follows the calling RPC or host HTTP request's cancellation signal. A disconnect while -draining removes only that request's pending hold, leaving successor and unrelated holds intact; -canceling activation does not cancel the mutations being drained. Completed holds use their TTL or -explicit release lifecycle. - -Holds and ordinary proxy leases are in-memory and do not survive daemon restart. Controllers must -reconnect and re-establish them; no persisted hold store is used. Local takeover is deferred: a -future host-global human-control fence must coexist with the local session's device claim, not -acquire it exclusively. +Human-control holds coexist with an open remote session. +`LeaseRegistry` owns them. +They use the same backend/provider/device contention key as device-aware leases. +The registry owns one lifecycle for hold heartbeat, expiry, lease preservation, and release refresh. +Release or expiry of the last hold restores the lease's existing inactivity TTL. +For expiry, renewal uses the hold's expiry instant. + +Tenant hold operations are ordinary daemon RPCs admitted through `request-admission.ts`. +Only the admitted lease supplies their device. +Host administration is a distinct loopback capability authenticated with the daemon token, never a tenant credential. +Tenants cannot modify host holds. + +Mutation admission derives from existing recording effects, observation-class inventory, and observability semantics. +Takeover and lease heartbeats are exempt from the mutation fence, but retain ownership checks. +Unknown effects are mutations. +A pending activation fences new mutations. +Before it reports active, it drains mutations already admitted. +Advisory execution locks alone do not establish this guarantee for fresh sessions. + +Activation follows the calling RPC or host HTTP request's cancellation signal. +If disconnect occurs during drain, only that request's pending hold is removed. +Successor and unrelated holds remain intact. +Canceled activation does not cancel mutations under drain. +Completed holds use their TTL or explicit release lifecycle. + +Holds and ordinary proxy leases remain in memory and do not survive daemon restart. +Controllers must reconnect and establish them again. +No persisted hold store exists. +Local takeover is deferred. +A future host-global human-control fence must coexist with the local session's device claim instead of acquiring it exclusively. ## macOS app leases -A lease on the macOS host would rent the whole desktop, so the macOS platform rents one app instead. -A `macos-app` lease's device key is a bundle id, optionally pinned to one process -(`` or `@`). The scope would be self-chosen if a tenant could name it, so -only a host administrator allocates one, over the loopback `/admin/leases` route that uses the daemon -token like host holds; tenant `lease_allocate` refuses the backend. The host picks the lease id, a -repeated PUT renews it, and a PUT naming another scope for an existing id is refused rather than -rewritten. Heartbeat, expiry, and the loss on daemon restart are those of any lease, except that a -tenant heartbeat or request cannot renew it for longer than the window of the host's last PUT, and -a tenant cannot release it: `lease_release` is refused with `MACOS_APP_LEASE_HOST_OWNED`. The lease -ends by the host's `DELETE /admin/leases/`, by expiry, by the loss on daemon restart, or -when a session it holds closes and the host set `retainOnClose` to false. - -Request admission confines every request admitted under the lease, so `batch` steps and `replay` -actions are confined when they re-enter it: an allow list of commands, the ones whose command -registry descriptor declares `appLease: 'allowed'` (later commands are refused, and of the commands -lease admission otherwise exempts only `lease_heartbeat` and `lease_release` declare it), -`open` and `close` of the leased bundle only, the `app` surface only, window-only screenshots, no -input that names a host path or launches beside the app, and an existing session that is the leased -app for every request but `open` and the `batch` envelope, so a -request naming no session cannot fall back to the host Mac. `open` requires the native app backend (ADR 0031), because XCTest -posts screen events that can land outside the app's window. A pid-pinned lease is checked against the -running process before each admitted request. A session opened under the lease holds its app, not -the Mac: it takes no host device claim, and other app-leased sessions on the same Mac do not conflict -with it, so one daemon serves several leased apps beside the host's own sessions. - -These rules bind a request that names the lease or runs in its session. A daemon policy -`leases.require` (ADR 0029) refuses requests that name no lease. The proxy token is one credential -for every client, so a host serving several clients through one proxy authenticates each client and -sets its tenant, session isolation, and lease on every request it forwards. +A lease on the macOS host would rent the whole desktop. +The macOS platform therefore rents one app instead. +A `macos-app` lease uses a bundle id as its device key, optionally pinned to one process: `` or `@`. +If tenants could name this scope, they would choose their own confinement. +Thus, only a host administrator allocates these leases through loopback `/admin/leases` with the daemon token, as for host holds. +Tenant `lease_allocate` refuses this backend. +The host selects the lease id. +A repeated PUT renews it. +If a PUT names another scope for an existing id, it is refused instead of changing scope. + +Heartbeat, expiry, and restart loss follow ordinary lease behavior, with two exceptions: + +- A tenant heartbeat or request cannot renew beyond the window of the host's last PUT. +- A tenant cannot release the lease. `lease_release` returns `MACOS_APP_LEASE_HOST_OWNED`. + +The lease ends through: + +- Host `DELETE /admin/leases/`. +- Expiry. +- Daemon restart. +- Close of a session it holds, if the host set `retainOnClose` to false. + +Request admission confines every admitted request under the lease. +This includes `batch` steps and `replay` actions when they enter admission again. +Confinement requires: + +- Commands whose registry descriptor declares `appLease: 'allowed'`. + Later commands are refused. + Among commands otherwise exempt from lease admission, only `lease_heartbeat` and `lease_release` declare it. +- `open` and `close` limited to the leased bundle. +- Only the `app` surface. +- Only window screenshots. +- No input that names a host path or launches beside the app. +- An existing session for the leased app on every request except `open` and the `batch` envelope. + +Thus, a request without a session cannot fall back to the host Mac. +`open` requires the native app backend from ADR 0031. +XCTest posts screen events that can reach outside the app window. +Before each admitted request, a pid-pinned lease is verified against the running process. +A session under the lease holds its app, not the Mac. +It takes no host device claim. +Other app-leased sessions on the same Mac do not conflict with it. +Thus, one daemon serves several leased apps beside the host's own sessions. + +These rules bind a request that names the lease or runs in its session. +Daemon policy `leases.require` in ADR 0029 refuses requests without a lease. +The proxy token is one credential shared by every client. +A host that serves several clients through one proxy therefore authenticates each client. +It sets tenant, session isolation, and lease on every forwarded request. From 03c8bbe141e1757be3492a65280204445f7eec93 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 16:59:59 +0200 Subject: [PATCH 11/55] docs(adr): remove completed ADR 0008 process history --- docs/adr/0008-command-descriptor-registry.md | 22 +++++--------------- 1 file changed, 5 insertions(+), 17 deletions(-) diff --git a/docs/adr/0008-command-descriptor-registry.md b/docs/adr/0008-command-descriptor-registry.md index 25bfeeec96..599f31fd1f 100644 --- a/docs/adr/0008-command-descriptor-registry.md +++ b/docs/adr/0008-command-descriptor-registry.md @@ -4,17 +4,15 @@ Accepted +Completed process history is in git history. + > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** `CommandDescriptor` remains > the command declaration root. Device-command capability buckets evolve into typed required and > preferred runtime use joined with leaf/device/provider-specific runtime facts; inventory commands > declare inventory use instead of fabricating a device binding. The descriptor's other derived > projections remain accepted. > -> **Parity gate retired (2026-09).** The migration rule below — assert each derived table -> byte-for-byte against its hand-authored table before deleting the hand table — was followed, and -> completing it removed the gate's second operand. Each hand table went away as its projection was -> inverted out: #907 daemon routes, #908 capability matrix, #909 batch allowlist, #1084 -> daemon-client timeout lists, #1137 MCP exposure list. No parity test exists. Do not add one or +> No parity test exists. Do not add one or > look for one; the totality invariants are now carried by compile-time guards > (`CommandOwnerFileClaimsAreComplete`) and by literal pin-set tests that compare a name set rather > than two tables. @@ -77,18 +75,10 @@ descriptor's `execute`. The descriptor is the prerequisite for typed per-command deletes the `src/client-types.ts` mirror) and supplies the capability facet the platform-plugin work (ADR 0009) hooks into. -Migration is **strangler-fig and sequential** — never a big-bang: - -1. Introduce the `commandRegistry` as the root and **invert the import graph** so `command-catalog`, - `capabilities`, `daemon-command-registry`, and `batch-policy` become leaves that derive from it. -2. Promote each family's facet to a full `CommandDescriptor`, family by family. -3. Replace the dispatch `switch` with the registry-driven total map, arm by arm. - -Each derived table must be asserted **byte-for-byte equivalent** to the hand-authored table by a parity test -**before** the hand table is deleted. The principal risk is the import-cycle inversion: `command-catalog.ts` +The principal risk is the import-cycle inversion: `command-catalog.ts` has ~95 importers and the family facet currently imports `AgentDeviceClient`, so the descriptor module must own the `Input`/`Result` types and the client must be derived as a view type, enforced by a lint boundary. -As of 2026-07, the descriptor registry is live for command identity (`PUBLIC_COMMANDS`, +The descriptor registry is live for command identity (`PUBLIC_COMMANDS`, `INTERNAL_COMMANDS`, and local CLI names), daemon registry traits, capability matrix, structured-batch allowlist, daemon-client timeout policy, MCP exposure list, capability-checked CLI command list, post-action observation traits, and the platform dispatch @@ -103,5 +93,3 @@ daemon responses still expose dynamic payloads or whose closed result contracts modules must move those contracts before their client methods can stop returning `CommandRequestResult`. -This ADR owns the decision and its constraints; the roadmap that prototyped it has been retired, with -the delivered end-state enforced by the descriptor projections below. From 28bb6b042a8e4b9e00b07964f5b0052e469168d2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:00:00 +0200 Subject: [PATCH 12/55] docs(adr): write ADR 0008 in ASD-STE100 --- docs/adr/0008-command-descriptor-registry.md | 175 +++++++++++-------- 1 file changed, 103 insertions(+), 72 deletions(-) diff --git a/docs/adr/0008-command-descriptor-registry.md b/docs/adr/0008-command-descriptor-registry.md index 599f31fd1f..b6f3e1eeb8 100644 --- a/docs/adr/0008-command-descriptor-registry.md +++ b/docs/adr/0008-command-descriptor-registry.md @@ -6,90 +6,121 @@ Accepted Completed process history is in git history. -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** `CommandDescriptor` remains -> the command declaration root. Device-command capability buckets evolve into typed required and -> preferred runtime use joined with leaf/device/provider-specific runtime facts; inventory commands -> declare inventory use instead of fabricating a device binding. The descriptor's other derived -> projections remain accepted. +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** `CommandDescriptor` remains the root for command declarations. +> Device-command capability buckets evolve into typed required and preferred runtime use with leaf/device/provider-specific runtime facts. +> Inventory commands declare inventory use instead of a fabricated device binding. +> The descriptor's other derived projections remain accepted. > -> No parity test exists. Do not add one or -> look for one; the totality invariants are now carried by compile-time guards -> (`CommandOwnerFileClaimsAreComplete`) and by literal pin-set tests that compare a name set rather -> than two tables. +> No parity test exists. +> Do not add or search for one. +> Compile-time guards (`CommandOwnerFileClaimsAreComplete`) and literal pin-set tests now enforce totality. +> Those tests compare name sets instead of two tables. + +## Rules at a glance + +- Compose one `CommandDescriptor` per command from facets owned by their domains. +- Derive consumer tables through pure projections. +- Keep shared narrow command traits on the descriptor. +- Keep client `invoke` and in-daemon `execute` distinct across the process boundary. +- Preserve all four invariants from ADR 0003. +- The descriptor module must own `Input`/`Result` types. Derive the client as a view type under a lint boundary. ## Context -A command's identity is restated, by hand, across roughly ten tables that must stay aligned by -convention: `PUBLIC_COMMANDS` (`src/command-catalog.ts`), the per-command metadata and family facets -(`src/commands/**`), the capability matrix (`src/core/capabilities.ts`), the daemon command registry -(`packages/daemon/src/core/daemon-command-registry.ts`, ADR 0003), the structured-batch allowlist -(`src/batch-policy.ts`), the MCP exposure sets, the Node client interface and impl (`src/client-types.ts`, -`src/client.ts`), and the generic-dispatch `switch` (`src/core/dispatch.ts`, whose `default: throw` makes a -missing or renamed command a runtime error, not a compile error). Adding one command touches ~24 files, the -argument shape is (de)serialized ~4 times, and the gesture set is retyped in three places. +Roughly ten manually maintained tables repeat command identity and require alignment by convention: + +- `PUBLIC_COMMANDS` in `src/command-catalog.ts`. +- Per-command metadata and family facets in `src/commands/**`. +- Capability matrix in `src/core/capabilities.ts`. +- Daemon command registry in `packages/daemon/src/core/daemon-command-registry.ts` (ADR 0003). +- Structured-batch allowlist in `src/batch-policy.ts`. +- MCP exposure sets. +- Node client interface and implementation in `src/client-types.ts` and `src/client.ts`. +- Generic-dispatch `switch` in `src/core/dispatch.ts`. -The codebase already proves the cure works for part of this: the `CommandFamilyFacet` -(`src/commands/family/`) derives the MCP tools, the CLI schema, and the batch writer from a single array. -It simply stops at the command-surface boundary; everything past it is hand-maintained. +The switch's `default: throw` makes a missing or renamed command a runtime error instead of a compile error. +One new command changes approximately 24 files. +Its argument shape is serialized or deserialized approximately four times. +Three places repeat the gesture set. -ADR 0003 deliberately separated daemon route/policy into its own internally-owned registry with a small -predicate interface, and its 2026-06 update set four invariants that any single-declaration/derivation -model must preserve. This ADR is that model. +`CommandFamilyFacet` in `src/commands/family/` already proves that derivation works for part of this problem. +It derives MCP tools, CLI schema, and batch writer from one array. +However, derivation stops at the command-surface boundary. +Beyond it, tables remain manually maintained. + +ADR 0003 deliberately gives daemon route/policy a separately owned internal registry with a small predicate interface. +Its 2026-06 update defines four invariants for any single-declaration derivation model. +This ADR defines that model. ## Decision -Introduce one `CommandDescriptor` per command that **composes facets owned by their domains** and from which -every consumer table is **derived** by pure projection: - -- The descriptor composes catalog identity, command-family surface projection hooks - (`src/commands/**` remains the owner of CLI schema/reader and executable surface metadata), a - `capability` facet, a `daemon` facet (route + request-policy traits shaped by ADR 0003), platform - dispatch membership, response data transforms, and typed-result hooks. -- Narrow command traits that affect multiple projections, such as interaction post-action observation - (`--settle` / `--verify`), live on the descriptor so CLI flags, command metadata, timeout policy, and - tests derive from one fact instead of repeating command-name lists. -- The public catalog, capability matrix, daemon command registry, batch allowlist, MCP tool list, CLI - schema exposure, and descriptor-backed Node client surface become pure projections of the descriptor set. The - `src/core/dispatch.ts` `switch` is replaced by a total map keyed on the command-name union, so a missing - handler is a compile error. -- The cross-process `invoke` (client) and in-daemon `execute` seams stay distinct; the process boundary is - never collapsed. - -This **composes with**, and is bound by, ADR 0003's four invariants: daemon-owned declaration (never inlined -into the public surface), the predicate interface unchanged, no leakage of daemon-only traits into public -projections, and one declaration per concern enforced by the type system. +Introduce one `CommandDescriptor` per command. +It **composes facets owned by their domains**. +Every consumer table is **derived** through pure projection: + +- The descriptor composes catalog identity, command-family surface projection hooks, `capability`, `daemon`, platform dispatch membership, response data transforms, and typed-result hooks. + `src/commands/**` retains CLI schema/reader and executable surface metadata ownership. + ADR 0003 defines the daemon facet's route and request-policy traits. +- Narrow command traits that affect multiple projections belong to the descriptor. + Interaction post-action observation (`--settle` / `--verify`) is one example. + CLI flags, command metadata, timeout policy, and tests derive from one fact instead of repeated command-name lists. +- These surfaces become pure projections of the descriptor set: + - Public catalog. + - Capability matrix. + - Daemon command registry. + - Batch allowlist. + - MCP tool list. + - CLI schema exposure. + - Descriptor-backed Node client surface. + A total map keyed on the command-name union replaces the `src/core/dispatch.ts` `switch`. + A missing handler therefore causes a compile error. +- Cross-process client `invoke` and in-daemon `execute` remain distinct. + The process boundary is never removed. + +This design follows all four ADR 0003 invariants: + +1. Daemon-owned declaration, never inline in the public surface. +2. Unchanged predicate interface. +3. No daemon-only traits in public projections. +4. One declaration per concern, enforced by types. ## Alternatives Considered -- Keep the hand-synced tables: no migration risk, but it is the status quo this ADR exists to remove — - ~24-file cost per command and drift kept in check only by convention and tests. -- A single flat public descriptor with daemon fields inlined: re-contaminates the public command surface - with daemon-only policy, which is exactly what ADR 0003 (and its update) forbid. -- Build-time code generation: a real option, but runtime derivation with `as const satisfies` keeps the - source of truth in type-checked TypeScript with no separate build step or generated artifacts to review. +- **Keep manually synchronized tables.** This avoids migration risk. + However, each command still costs approximately 24 changed files, with alignment enforced only by convention and tests. +- **Use one flat public descriptor with inline daemon fields.** This puts daemon-only policy in the public command surface. + ADR 0003 and its update explicitly forbid this ownership combination. +- **Generate code at build time.** This is a valid option. + However, runtime derivation with `as const satisfies` retains the source of truth in type-checked TypeScript. + It requires no separate build step or generated artifacts for review. ## Consequences -Adding a plain command touches ~1–2 files; per-platform behavior remains N implementations behind the -descriptor's `execute`. The descriptor is the prerequisite for typed per-command results (ADR 0010, which -deletes the `src/client-types.ts` mirror) and supplies the capability facet the platform-plugin work -(ADR 0009) hooks into. - -The principal risk is the import-cycle inversion: `command-catalog.ts` -has ~95 importers and the family facet currently imports `AgentDeviceClient`, so the descriptor module must -own the `Input`/`Result` types and the client must be derived as a view type, enforced by a lint boundary. -The descriptor registry is live for command identity (`PUBLIC_COMMANDS`, -`INTERNAL_COMMANDS`, and local CLI names), daemon registry traits, capability matrix, -structured-batch allowlist, daemon-client timeout policy, MCP exposure list, -capability-checked CLI command list, post-action observation traits, and the platform dispatch -command set (including the remaining dispatch-only alias `read`). Command-family -surface metadata still lives under `src/commands/**`, where the CLI grammar and client-backed -executors already live, but it is coherence-guarded against the descriptor CLI catalog so new -surface names cannot drift from the descriptor root. - -The remaining deferred work is the public Node-client typed-result narrowing tracked outside this -registry migration in [#1153](https://github.com/callstack/agent-device/issues/1153): commands whose -daemon responses still expose dynamic payloads or whose closed result contracts live in runtime -modules must move those contracts before their client methods can stop returning -`CommandRequestResult`. - +A plain new command changes approximately 1–2 files. +Per-platform behavior remains N implementations behind the descriptor's `execute`. +The descriptor is a prerequisite for typed per-command results in ADR 0010, which deletes the `src/client-types.ts` mirror. +It also supplies the capability facet used by the platform-plugin work in ADR 0009. + +Import-cycle inversion is the principal risk. +`command-catalog.ts` has approximately 95 importers, and the family facet currently imports `AgentDeviceClient`. +Thus, the descriptor module must own `Input`/`Result` types. +The client must be derived as a view type, enforced by a lint boundary. +The descriptor registry supplies: + +- Command identity: `PUBLIC_COMMANDS`, `INTERNAL_COMMANDS`, and local CLI names. +- Daemon registry traits. +- Capability matrix. +- Structured-batch allowlist. +- Daemon-client timeout policy. +- MCP exposure list. +- Capability-checked CLI command list. +- Post-action observation traits. +- Platform dispatch command set, including the remaining dispatch-only alias `read`. + +Command-family surface metadata remains under `src/commands/**`, beside CLI grammar and client-backed executors. +Coherence checks compare it with the descriptor CLI catalog. +Thus, new surface names cannot differ from the descriptor root. + +Public Node-client typed-result narrowing remains deferred outside this registry migration in [#1153](https://github.com/callstack/agent-device/issues/1153). +Some commands still expose dynamic daemon response payloads or retain closed result contracts in runtime modules. +Before their client methods can stop returning `CommandRequestResult`, those contracts must move. From b43a7cd34246a3094fec064f8f77501de44cac35 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:01:03 +0200 Subject: [PATCH 13/55] docs(adr): remove completed ADR 0009 process history --- docs/adr/0009-apple-platform-consolidation.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/docs/adr/0009-apple-platform-consolidation.md b/docs/adr/0009-apple-platform-consolidation.md index a77ad70c29..bb82f89127 100644 --- a/docs/adr/0009-apple-platform-consolidation.md +++ b/docs/adr/0009-apple-platform-consolidation.md @@ -4,6 +4,8 @@ Accepted +Completed process history is in git history. + > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** The Apple family and explicit > `AppleOS` leaf axis remain accepted. The shallow `PlatformPlugin` shape and root > `src/platforms/**` location are superseded by the lazy platform-module registry and private @@ -29,10 +31,8 @@ Model Apple OSes with an **`AppleOS` discriminant** (`ios | ipados | tvos | watc under a single `apple` Platform — **not** six `Platform` literals. The OS-agnostic Apple engine consolidates under `packages/platform-apple/src/core/`, with genuinely per-OS code in `packages/platform-apple/src/os//` leaves; -the Apple plugin is the first instance of the platform-plugin registry (the platform axis of the -completed perfect-shape refactor; the registry and enforcement paths below are authoritative). -Per-OS capability differences become data keyed by `AppleOS`. The additive, -non-breaking `appleOs` discriminant — the groundwork for this — shipped in #896. +the Apple plugin is the first instance of the platform-plugin registry. +Per-OS capability differences become data keyed by `AppleOS`. ## Alternatives Considered @@ -54,15 +54,14 @@ net-new work (XCUITest supports it — a profile row, a build case, `#if os(visi filter, plus real spatial-input QA); watchOS is an explicit **unsupported sentinel** because XCUITest cannot drive watchOS UI. macOS stays a distinct AppKit leaf (its helper binary and menubar/desktop surface model are preserved). The tvOS focus-only interaction contract (no coordinate `tap`) must not be flattened across OSes, -and snapshot fidelity is uneven (the deep-RN AX-server fallback is iOS-simulator-only). The internal -`Platform` collapse of `ios`+`macos` into `apple` was the last, highest-diff step; public leaf output +and snapshot fidelity is uneven (the deep-RN AX-server fallback is iOS-simulator-only). Public leaf output remains a separate compatibility projection. This composes with ADR 0008 (the descriptor's capability facet) and ADR 0003. -Implementation status as of 2026-08: +Support boundary: -- Shipped: the internal `Platform` collapse to `apple`; additive `appleOs` groundwork; the shared +- the internal `Platform` collapse to `apple`; additive `appleOs` groundwork; the shared Apple engine under `packages/platform-apple/src/core`; macOS leaf files under `packages/platform-apple/src/os/macos`; a dedicated tvOS leaf under `packages/platform-apple/src/os/tvos`; @@ -82,6 +81,5 @@ Implementation status as of 2026-08: `PUBLIC_PLATFORMS`; internal family ownership must not leak into that projection. - Deferred: net-new visionOS spatial-input QA. -This ADR owns the Apple family/leaf decision. The Phase 3 platform-plugin umbrella (#972) is closed; -ADR 0019 now owns the staged package/runtime seam. Net-new visionOS spatial-input QA remains a +This ADR owns the Apple family/leaf decision. ADR 0019 owns the staged package/runtime seam. Net-new visionOS spatial-input QA remains a separate follow-up outside the consolidation. From a024cadd345b0bca930273fa0fa5dadfe20078d4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:01:03 +0200 Subject: [PATCH 14/55] docs(adr): write ADR 0009 in ASD-STE100 --- docs/adr/0009-apple-platform-consolidation.md | 162 +++++++++++------- 1 file changed, 98 insertions(+), 64 deletions(-) diff --git a/docs/adr/0009-apple-platform-consolidation.md b/docs/adr/0009-apple-platform-consolidation.md index bb82f89127..895732ff15 100644 --- a/docs/adr/0009-apple-platform-consolidation.md +++ b/docs/adr/0009-apple-platform-consolidation.md @@ -6,80 +6,114 @@ Accepted Completed process history is in git history. -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** The Apple family and explicit -> `AppleOS` leaf axis remain accepted. The shallow `PlatformPlugin` shape and root -> `src/platforms/**` location are superseded by the lazy platform-module registry and private -> `@agent-device/platform-*` packages. Execution cuts over command-atomically; shared physical -> mechanics move only through ADR 0019's legal injected transition or after their last legacy user. +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** The Apple family and explicit `AppleOS` leaf axis remain accepted. +> The lazy platform-module registry and private `@agent-device/platform-*` packages supersede the shallow `PlatformPlugin` shape and root `src/platforms/**` location. +> Each execution migration covers one complete command. +> Shared physical mechanics move only through ADR 0019's permitted injected transition or after their last legacy consumer. + +## Rules at a glance + +- Use one `apple` Platform with an `AppleOS` discriminant, not six Platform literals. +- Keep shared Apple mechanics in `packages/platform-apple/src/core/` and per-OS code in `packages/platform-apple/src/os//`. +- Express per-OS capability differences as data keyed by `AppleOS`. +- Preserve macOS's distinct AppKit leaf and tvOS's focus-only interaction contract. +- Keep watchOS as an unsupported sentinel. Discovery never produces it. +- Keep public `ios`/`macos` output separate from internal family ownership. +- Do not claim public-command coverage for visionOS or watchOS. visionOS spatial-input QA remains deferred. ## Context -Apple support is modeled asymmetrically and is physically smeared across an `ios` directory that is really -the shared Apple engine. `Platform` carries `ios` and `macos` as separate literals, but tvOS is not a -platform at all — it is `platform: 'ios' + target: 'tv'`, with the OS name reconstructed late and lossily by -`resolveApplePlatformName` (`src/kernel/device.ts`). Meanwhile ~697 LOC of macOS code lives inside -`src/platforms/ios/`, `src/platforms/macos/devices.ts` is a 19-line stub that the iOS discovery imports (the -dependency arrow points backwards), and the Apple interactor in `src/core/interactors/apple.ts` reaches into -`platforms/ios`. A four-investigator survey found that ~85% of `platforms/ios` (the runner stack, -tool-provider, discovery, snapshot, screenshot, perf, debug-symbols) is already OS-agnostic, and the XCTest -runner already builds `ios | macos | tvos` from one Xcode project. Distinguishing or adding an Apple OS today -is therefore costly out of proportion to the actual work, and iPadOS/visionOS/watchOS are unmodeled. +Apple support uses an asymmetric model. +An `ios` directory contains much of the shared Apple engine. +`Platform` has separate `ios` and `macos` literals, but tvOS uses `platform: 'ios' + target: 'tv'`. +`resolveApplePlatformName` in `src/kernel/device.ts` reconstructs the OS name late and loses information. +Approximately 697 LOC of macOS code resides in `src/platforms/ios/`. +`src/platforms/macos/devices.ts` is a 19-line stub imported by iOS discovery, which reverses the intended dependency direction. +The Apple interactor in `src/core/interactors/apple.ts` also imports from `platforms/ios`. + +A survey by four investigators found approximately 85% of `platforms/ios` already OS-neutral: + +- Runner stack. +- Tool-provider. +- Discovery. +- Snapshot. +- Screenshot. +- Perf. +- Debug-symbols. + +The XCTest runner already builds `ios | macos | tvos` from one Xcode project. +Thus, distinguishing or adding an Apple OS costs more than its actual implementation requires. +iPadOS, visionOS, and watchOS have no model. ## Decision -Model Apple OSes with an **`AppleOS` discriminant** (`ios | ipados | tvos | watchos | visionos | macos`) -under a single `apple` Platform — **not** six `Platform` literals. The OS-agnostic Apple engine consolidates -under `packages/platform-apple/src/core/`, with genuinely per-OS code in -`packages/platform-apple/src/os//` leaves; -the Apple plugin is the first instance of the platform-plugin registry. +Model Apple OSes with an **`AppleOS` discriminant** (`ios | ipados | tvos | watchos | visionos | macos`) under one `apple` Platform. +Do not use six `Platform` literals. +Consolidate the OS-neutral Apple engine under `packages/platform-apple/src/core/`. +Keep genuinely per-OS code in `packages/platform-apple/src/os//` leaves. +The Apple plugin is the first instance of the platform-plugin registry. Per-OS capability differences become data keyed by `AppleOS`. ## Alternatives Considered -- Promote each Apple OS to its own `Platform` literal: rejected. `DeviceTarget` (`mobile | tv | desktop`) is - already cross-platform (Android TV uses `target: 'tv'`), so a `tvos` literal collides with the form-factor - axis; it would also force the ~15 `isApplePlatform` and ~52 `macos` branch sites to enumerate six literals - and break the single-bucket `apple` capability/selector model that already works. -- Keep the status quo: rejected — the tvOS/macOS asymmetry, the mislabeled macOS code, and the lossy - target→OS inference persist, with no path to iPadOS/visionOS. -- Exclude macOS from the consolidation: rejected. macOS is already entangled — it builds via the same XCUITest - project and ~697 LOC of its code already lives inside `platforms/ios`. Excluding it would leave the - mislabel in place; including it as a distinct AppKit leaf normalizes it without homogenizing it. +- **Give each Apple OS its own `Platform` literal.** Rejected. + `DeviceTarget` (`mobile | tv | desktop`) already spans platforms. + Android TV uses `target: 'tv'`, so a `tvos` literal would conflict with the form-factor axis. + Approximately 15 `isApplePlatform` sites and 52 `macos` branches would need to enumerate six literals. + This would also break the working single-bucket `apple` capability/selector model. +- **Keep the existing structure.** Rejected. + It retains tvOS/macOS asymmetry, mislabeled macOS code, and lossy target→OS inference without a path to iPadOS/visionOS. +- **Exclude macOS from consolidation.** Rejected. + macOS already builds through the same XCUITest project, and approximately 697 LOC resides in `platforms/ios`. + Exclusion would retain the incorrect label. + A distinct AppKit leaf normalizes ownership while preserving macOS-specific behavior. ## Consequences -Adding a first-class Apple OS becomes cheap: a leaf module plus a runner-profile row. iOS/iPadOS/tvOS/macOS -are mostly relocate-and-rename (the engine never needed to know which Apple OS it drives); visionOS is scoped -net-new work (XCUITest supports it — a profile row, a build case, `#if os(visionOS)`, a widened discovery -filter, plus real spatial-input QA); watchOS is an explicit **unsupported sentinel** because XCUITest cannot -drive watchOS UI. macOS stays a distinct AppKit leaf (its helper binary and menubar/desktop surface model are -preserved). The tvOS focus-only interaction contract (no coordinate `tap`) must not be flattened across OSes, -and snapshot fidelity is uneven (the deep-RN AX-server fallback is iOS-simulator-only). Public leaf output -remains a separate compatibility projection. - -This composes with ADR 0008 (the descriptor's capability facet) and ADR 0003. - -Support boundary: - -- the internal `Platform` collapse to `apple`; additive `appleOs` groundwork; the shared - Apple engine under `packages/platform-apple/src/core`; macOS leaf files under - `packages/platform-apple/src/os/macos`; a dedicated tvOS leaf under - `packages/platform-apple/src/os/tvos`; - direct internal imports to the Apple modules; the per-`AppleOS` runtime facts (including - `packages/platform-apple/src/gesture-facts.ts`; the former capability projection was retired after - its predicates moved into request-bound facts); the watchOS **unsupported sentinel** (reserved in the `AppleOS` type and interactor-rejected — - XCUITest cannot drive watchOS UI — `isSupportedAppleDeploymentLeaf`, the Apple interactor, and - gesture admission reject it — never produced by discovery); and visionOS profile/build/discovery - plus simulator-deployment evidence. -- Decision-only support boundary: visionOS discovery and simulator deployment are supported and - unit-tested (`packages/platform-apple/src/inventory-classification.ts` and - `packages/platform-apple/src/deployment/runtime.test.ts`); app deployment is also admitted for - CoreDevice-backed physical devices, while XCTest-backed physical deployment is unsupported and - push remains simulator-only. No public-command coverage is claimed for visionOS or watchOS. This - boundary is recorded here without creating a command-coverage manifest for either leaf. -- Retained compatibility: the public wire still emits `ios`/`macos` leaves through - `PUBLIC_PLATFORMS`; internal family ownership must not leak into that projection. -- Deferred: net-new visionOS spatial-input QA. - -This ADR owns the Apple family/leaf decision. ADR 0019 owns the staged package/runtime seam. Net-new visionOS spatial-input QA remains a -separate follow-up outside the consolidation. +A first-class Apple OS needs only a leaf module and runner-profile row for the shared structure. +iOS/iPadOS/tvOS/macOS mostly require moves and renames because the engine does not need the specific Apple OS. +visionOS requires scoped new work because XCUITest supports it: + +- A profile row. +- A build case. +- `#if os(visionOS)`. +- A wider discovery filter. +- Real spatial-input QA. + +watchOS is an explicit **unsupported sentinel** because XCUITest cannot operate its UI. +macOS remains a distinct AppKit leaf with its helper binary and menubar/desktop surface model. +The tvOS focus-only interaction contract, with no coordinate `tap`, must remain distinct across OSes. +Snapshot fidelity is uneven. +The deep-RN AX-server fallback is iOS-simulator-only. +Public leaf output remains a separate compatibility projection. + +This design composes with ADR 0008's descriptor capability facet and ADR 0003. + +The support boundary includes: + +- Internal `Platform` ownership uses `apple`, with additive `appleOs`. +- The shared Apple engine resides under `packages/platform-apple/src/core`. +- macOS leaf files reside under `packages/platform-apple/src/os/macos`. +- A dedicated tvOS leaf resides under `packages/platform-apple/src/os/tvos`. +- Internal imports access Apple modules directly. +- Per-`AppleOS` runtime facts include `packages/platform-apple/src/gesture-facts.ts`. + The former capability projection's predicates now belong to request-bound facts. +- `AppleOS` reserves watchOS, but the interactor rejects it. + `isSupportedAppleDeploymentLeaf`, the Apple interactor, and gesture admission all reject watchOS. + Discovery never produces it. +- visionOS has profile/build/discovery support and simulator-deployment evidence. + +visionOS discovery and simulator deployment are supported and unit-tested in `packages/platform-apple/src/inventory-classification.ts` and `packages/platform-apple/src/deployment/runtime.test.ts`. +CoreDevice-backed physical devices also admit app deployment. +XCTest-backed physical deployment remains unsupported. +Push remains simulator-only. +No public-command coverage is claimed for visionOS or watchOS. +This ADR records that decision-only boundary without a command-coverage manifest for either leaf. + +The public wire retains `ios`/`macos` leaves through `PUBLIC_PLATFORMS`. +Internal family ownership must not enter that projection. +New visionOS spatial-input QA remains deferred. + +This ADR owns the Apple family/leaf decision. +ADR 0019 owns the staged package/runtime boundary. +New visionOS spatial-input QA remains a separate follow-up outside consolidation. From d5e33d32495c04c7b3474034eb5d44d1d61b4194 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:01:03 +0200 Subject: [PATCH 15/55] docs(adr): write ADR 0010 in ASD-STE100 --- docs/adr/0010-error-system.md | 134 +++++++++++++++++++++------------- 1 file changed, 84 insertions(+), 50 deletions(-) diff --git a/docs/adr/0010-error-system.md b/docs/adr/0010-error-system.md index eb24ec378d..1de13213d0 100644 --- a/docs/adr/0010-error-system.md +++ b/docs/adr/0010-error-system.md @@ -3,62 +3,96 @@ - Status: accepted - Date: 2026-07-03 -> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Existing normalization and -> cause-preservation rules remain accepted. When an operation and binding/resource cleanup both -> fail, the operation remains primary and cleanup is structured secondary diagnostic evidence; -> cleanup-only failure surfaces normally. +> **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Existing normalization and cause-preservation rules remain accepted. +> If an operation and binding/resource cleanup both fail, the operation remains primary. +> Cleanup becomes structured secondary diagnostic evidence. +> A cleanup-only failure returns normally as a failure. + +## Rules at a glance + +- Every reachable thrown error is an `AppError`. Reserve bare `Error` for provably unreachable invariants. +- For input validation, use `INVALID_ARGS`. For unknown caught errors, use `asAppError(err, fallbackCode)`. +- Use the most specific `KnownAppErrorCode`. Put machine-dispatchable subclasses in `details.reason`. +- Supply a hint only when the default would mislead. Preserve existing hints and causes through wrappers. +- Every consumer must carry at least code, message, and hint. Rehydration helpers copy all error fields. +- Failed requests always flush diagnostics. Messages must not contain raw stderr dumps or secrets. ## Context -The error system is centralized in `src/kernel/errors.ts` (`AppError`, `normalizeError`, -`defaultHintForCode`, `retriableForErrorCode`) but has ~1,200 construction sites across the CLI, -daemon, and platform layers. A July 2026 audit (code sweep + iOS/Android dogfooding) found the -kernel sound while call-site quality drifted: internal validation thrown as bare `Error` -(surfacing as `UNKNOWN` with a misleading hint), semantically wrong codes (`COMMAND_FAILED` for -busy devices, `INVALID_ARGS` for expired server-side resources), missing hints on the most common -agent failures (selector/ref misses), and consumers that drop fields (MCP tool errors carried only -`message`). Nothing documented the conventions, so each new site re-decided them. +`src/kernel/errors.ts` centralizes `AppError`, `normalizeError`, `defaultHintForCode`, and `retriableForErrorCode`. +However, approximately 1,200 sites construct errors across CLI, daemon, and platform layers. +A July 2026 audit combined code examination with iOS/Android use. +It found a sound kernel but inconsistent call sites: + +- Internal validation threw bare `Error`, which appeared as `UNKNOWN` with a misleading hint. +- Some codes had incorrect semantics, such as `COMMAND_FAILED` for busy devices or `INVALID_ARGS` for expired server-side resources. +- Common selector/ref failures lacked hints. +- Consumers lost fields, such as MCP tool errors that carried only `message`. + +Without documented conventions, each new site selected its own behavior. ## Decision -1. **Every thrown error a user or agent can reach is an `AppError`.** Bare `throw new Error(...)` - is reserved for provably unreachable invariants. Input validation — including daemon-side - decoding of command input — throws `AppError('INVALID_ARGS', ...)`. Coercion of unknown caught - errors uses `asAppError(err, fallbackCode)`, which preserves the cause chain; do not hand-roll - `err instanceof AppError ? err : new AppError(...)`. -2. **Code selection.** Use the most specific `KnownAppErrorCode`; `COMMAND_FAILED` is for genuine - runtime failures of a well-formed request, never a catch-all for capability gaps - (`UNSUPPORTED_OPERATION`), contention (`DEVICE_IN_USE`, the only code retriable by default; - the cross-worktree device-claim path overrides it to `retriable: false`), or ambiguity - (`AMBIGUOUS_MATCH`). New codes are added to the union deliberately; machine-dispatchable - sub-classification rides in `details.reason` (the lease registry is the model). -3. **Hints answer "what should the agent run next".** A hint is required wherever the per-code - default from `defaultHintForCode` would mislead; it is omitted where the default suffices — - mass-adding boilerplate hints is worse than the default. Shared failure modes get shared hint - constants next to the code that detects them (`selectorFailureHint`, `STALE_REF_HINT` in - `packages/selectors/src/internal/resolve.ts`; `resolveIosDevicectlHint`; `bootFailureHint`), not copy-pasted - strings. Re-wraps preserve an existing hint rather than clobbering it. -4. **Wrapping external tool failures.** Prefer `exec.ts` errors as-is. A hand-rolled wrap of an - `allowFailure` result must carry `{ stdout, stderr, exitCode, processExitError: true }` so - `normalizeError` can surface the first meaningful stderr line instead of "tool exited with - code N". -5. **The wire error contract** is `code`, `message`, `hint`, `details` (redacted), `diagnosticId`, - `logPath`, plus optional typed signals `retriable` / `supportedOn` — the fields are absent - unless confidently known. Every consumer surface (CLI human, CLI `--json`, SDK, MCP, events - timeline) must carry code + message + hint at minimum; rehydration helpers - (`throwDaemonError`, `toDaemonHttpRpcError`) copy all fields, and `NormalizedError` hoists the - typed signals so `--json` consumers see them. -6. **Observability.** Failed requests always flush diagnostics: `diagnosticId` + ndjson `logPath` - accompany every error (daemon-side under the session state dir, client-side under - `~/.agent-device/logs`). Messages must not embed raw stderr dumps or secrets — structured - context belongs in `details`, which is redacted at normalize/write time. +1. **Every thrown error a user or agent can reach is an `AppError`.** + Reserve bare `throw new Error(...)` for provably unreachable invariants. + Input validation, including daemon-side command input decode, throws `AppError('INVALID_ARGS', ...)`. + For unknown caught errors, use `asAppError(err, fallbackCode)` to preserve the cause chain. + Do not construct a replacement through `err instanceof AppError ? err : new AppError(...)`. +2. **Code selection.** Use the most specific `KnownAppErrorCode`. + `COMMAND_FAILED` means a genuine runtime failure of a well-formed request. + Never use it as a general code for: + - Capability gaps: `UNSUPPORTED_OPERATION`. + - Contention: `DEVICE_IN_USE`. + - Ambiguity: `AMBIGUOUS_MATCH`. + Only `DEVICE_IN_USE` is retriable by default. + The cross-worktree device-claim path overrides it with `retriable: false`. + Add new codes to the union deliberately. + Put machine-dispatchable subclasses in `details.reason`, as the lease registry does. +3. **Hints answer "what should the agent run next".** + If the per-code default from `defaultHintForCode` would mislead, a hint is required. + If the default suffices, omit the hint. + Repeated boilerplate hints are worse than the default. + Shared failure modes use shared hint constants beside their detection code, instead of copied strings: + - `selectorFailureHint` and `STALE_REF_HINT` in `packages/selectors/src/internal/resolve.ts`. + - `resolveIosDevicectlHint`. + - `bootFailureHint`. + Wrappers preserve an existing hint instead of replacing it. +4. **Wrapping external tool failures.** Prefer unchanged errors from `exec.ts`. + A manual wrapper of an `allowFailure` result must carry `{ stdout, stderr, exitCode, processExitError: true }`. + Thus, `normalizeError` can expose the first meaningful stderr line instead of "tool exited with code N". +5. **The wire error contract** contains: + - `code`. + - `message`. + - `hint`. + - Redacted `details`. + - `diagnosticId`. + - `logPath`. + - Optional typed signals `retriable` / `supportedOn`. + Unless confidently known, typed signals are absent. + Every consumer surface must carry at least code, message, and hint: + - CLI human output. + - CLI `--json`. + - SDK. + - MCP. + - Events timeline. + Rehydration helpers `throwDaemonError` and `toDaemonHttpRpcError` copy all fields. + `NormalizedError` exposes typed signals at its top level for `--json` consumers. +6. **Observability.** Failed requests always flush diagnostics. + Every error carries `diagnosticId` and ndjson `logPath`. + Daemon diagnostics reside under the session state dir. + Client diagnostics reside under `~/.agent-device/logs`. + Messages must not contain raw stderr dumps or secrets. + Structured context belongs in `details`, which is redacted at normalization/write time. ## Consequences -- Agents can branch on `code`, retry on `retriable`, and follow `hint` without parsing prose; - wording can improve without breaking consumers (except strings pinned by help-text guarantees — - update those tests in lockstep). -- The known-code union stays small and meaningful; the widened `(string & {})` type still lets - runner/daemon codes pass through, so SDK consumers keep a `default` branch. -- New call sites have a documented bar: right code, contextual message naming the target, hint - only when the default misleads, cause preserved. +- Agents can select behavior by `code`, retry by `retriable`, and follow `hint` without prose parsing. + Wording can improve without breaking consumers, except for strings fixed by help-text guarantees. + When those strings change, update their tests together. +- The known-code union remains small and meaningful. + The wider `(string & {})` type still passes runner/daemon codes through, so SDK consumers retain a `default` branch. +- New call sites have documented requirements: + - Correct code. + - Contextual message that names the target. + - Hint only when the default misleads. + - Preserved cause. From 308742e692da524af72c372bc41343d84b5360a9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:01:17 +0200 Subject: [PATCH 16/55] docs(adr): remove completed ADR 0011 process history --- .../adr/0011-interaction-guarantee-contract.md | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/docs/adr/0011-interaction-guarantee-contract.md b/docs/adr/0011-interaction-guarantee-contract.md index 3b0a904140..f721e4b8b5 100644 --- a/docs/adr/0011-interaction-guarantee-contract.md +++ b/docs/adr/0011-interaction-guarantee-contract.md @@ -2,7 +2,9 @@ ## Status -Accepted (implemented through Layer 3, 2026-07-04: #1080, #1082–#1086, #1091, #1092; ambiguity contract amended 2026-08-07) +Accepted + +Completed process history is in git history. ## Context @@ -396,20 +398,6 @@ gesture-plan contract in [ADR 0013](0013-unified-gesture-plans.md). It remains a sibling of this element-targeting matrix: coordinate gestures do not acquire selector/ref guarantees merely because their native executor uses XCTest. -## Migration plan - -Each step lands green and independently useful: - -1. **Registry + gate** with an honest initial classification (waivers linked - to issues). No behavior change. -2. **Response builder consolidation** (`buildInteractionResponseData`) + the - hand-rolled-literal guard test. -3. **Golden fixture tables** + `TapPointPolicy` Swift extraction + the three - consumers (vitest, XCTest, fake runner). -4. **Contract scenario suite** + registry-driven coverage gate; port the - Bluesky fixtures from #1075's tests into the permanent tables. -5. **Descriptor timeout policy**; delete the two client-side command lists. - ## Alternatives considered - **Delete the fast paths** (single resolution spine): unacceptable — the From 5a88e5ff89a623eeec70abda30488af10fc1ebac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:04:15 +0200 Subject: [PATCH 17/55] docs(adr): write ADR 0011 in ASD-STE100 --- .../0011-interaction-guarantee-contract.md | 676 +++++++++--------- 1 file changed, 343 insertions(+), 333 deletions(-) diff --git a/docs/adr/0011-interaction-guarantee-contract.md b/docs/adr/0011-interaction-guarantee-contract.md index f721e4b8b5..9f39d394bb 100644 --- a/docs/adr/0011-interaction-guarantee-contract.md +++ b/docs/adr/0011-interaction-guarantee-contract.md @@ -6,63 +6,67 @@ Accepted Completed process history is in git history. +## Rules at a glance + +- Classify every dispatch-path × guarantee cell in the typed registry. +- Link each gap waiver to a tracking issue. Keep command-scoped claims within the path's command set. +- Give each TypeScript rule one shared implementation. Prove runner-side equivalents through golden fixtures. +- Prove every non-waived cell through a tagged contract scenario. +- Delegation-on-error does not prove success-path parity. +- Before native-ref execution, apply shared guards to the stored node. +- Never resend an Apple mutation that may have run. +- Distinct mutating selector subtrees fail with `AMBIGUOUS_MATCH`, except for the stated compatibility path. +- Keep synthesized coordinate gesture policy separate from element-targeting guarantees. + ## Context -Interaction commands (`press`/`click`/`fill`/`longpress`, and by extension the -read/wait surfaces built on the same resolution machinery) reach the device -through several dispatch paths, each of which chose its latency-vs-semantics -trade-offs independently: +Interaction commands reach the device through several dispatch paths. +These commands include `press`/`click`/`fill`/`longpress` and related read/wait surfaces that share resolution machinery. +Each path independently selected its latency and semantic compromises: | Path | What it is | Why it exists | | --- | --- | --- | | `runtime-selector` | daemon tree capture → `resolveSelectorChain` → guards → coordinate tap | full semantics | | `runtime-ref` | session snapshot → ref lookup → guards → coordinate tap | full semantics | | `target-drag` | independently resolve source + destination → guards → one pointer plan | dual-endpoint drag semantics | -| `direct-ios-selector` | selector sent to the XCTest runner, which queries and taps natively | saves a full snapshot round trip | -| `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | saves resolution round trips | -| `coordinate` | raw x/y tap | escape hatch; semantics intentionally minimal | -| replay/maestro variants | `allowNonHittableCoordinateFallback`, replay-heal | third-party compat | - -Over one week of fixes, every interaction bug we shipped or caught was the same -shape — a guarantee enforced on one path and silently absent on a sibling: - -- off-screen refusal existed for `@ref` targets but not selector targets, and - not in the runner's native tap (Bluesky closed drawer: `Tapped (-161, 265)` - reported as success; fixed in #1075); -- `--verify` evidence attached on `press @ref` but was dropped by - `fill @ref`'s hand-built response branch (#1064 review); -- adb failure classification enriched thrown errors but not - `allowFailure`-result errors, and wrapped `exec` but not semantic provider - methods (#1067 review); -- the user's wait budget bounded prepare/replay/snapshot request envelopes but - not `wait` itself, and `wait` sat on the daemon-reset timeout path while - `snapshot` — same failure mode — preserved the daemon (#1075). - -The guarantees themselves are small and well-implemented. What is missing is -any structure that *knows the full set of paths and the full set of -guarantees* and can therefore notice an unfilled cell. Reviews catch cells -only when a reviewer happens to hold both axes in their head; dogfooding -catches them one production incident at a time. +| `direct-ios-selector` | selector sent to the XCTest runner for native query and tap | avoids a full snapshot round trip | +| `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | avoids resolution round trips | +| `coordinate` | raw x/y tap | explicit minimal semantics | +| replay/maestro variants | `allowNonHittableCoordinateFallback`, replay-heal | third-party compatibility | + +During one week, every shipped or detected interaction bug had the same cause. +One path enforced a guarantee that another path silently omitted: + +- Offscreen refusal covered `@ref` targets but omitted selector targets and native runner taps. + A closed Bluesky drawer reported success as `Tapped (-161, 265)` (#1075). +- `press @ref` attached `--verify` evidence. + The manually constructed response for `fill @ref` omitted it (#1064 review). +- adb failure classification enriched thrown errors but omitted errors from `allowFailure` results. + It wrapped `exec` but omitted semantic provider methods (#1067 review). +- The user's wait budget bounded prepare/replay/snapshot envelopes but omitted `wait` itself. + A `wait` timeout reset the daemon, while the same failure in `snapshot` preserved it (#1075). + +The guarantees are small and well implemented. +The missing structure enumerates every path and every guarantee to detect an unclassified cell. +Reviews otherwise depend on simultaneous knowledge of both axes. +Live use detects the gaps only after incidents. ## Decision -Make the path × guarantee matrix a first-class, machine-checked artifact, with -three enforcement layers. Types enforce **completeness** of declarations, -shared implementations prevent **drift**, and generated test coverage enforces -**truth**. +Make the path × guarantee matrix a machine-checked artifact with three enforcement layers. +Types enforce declaration **completeness**. +Shared implementations prevent **drift**. +Generated test coverage enforces **truth**. ### Layer 1 — Declare: the matrix as a typed registry with a gate -Layer 1 is an **honesty/completeness gate, not a truth gate**: it proves that -every path has declared a stance on every guarantee and that the referenced -implementations exist. It does not prove the declared behavior — behavioral -parity only starts once the fixture tables (Layer 2) and contract scenarios -(Layer 3) land. Landing Layer 1 alone still changes the failure mode: an -unwatched cell becomes impossible; an unproven cell is at least a visible, -owned claim. +Layer 1 proves **honesty/completeness**, not behavior. +It proves that every path declares a position on every guarantee and that referenced implementations exist. +Behavioral parity additionally requires Layer 2 fixture tables and Layer 3 contract scenarios. +Layer 1 alone prevents an undeclared cell. +An unproven cell remains a visible claim with an owner. -`packages/contracts/src/interaction-guarantees.ts` declares both axes and requires -every cell to be classified: +`packages/contracts/src/interaction-guarantees.ts` declares both axes and requires classification of every cell: ```ts export const INTERACTION_GUARANTEES = [ @@ -77,19 +81,21 @@ export const INTERACTION_GUARANTEES = [ ] as const; ``` -Cells may be **command-scoped** via `appliesTo` when a guarantee only exists -for a subset of the path's commands — e.g. `verifyEvidence` applies to -`press`/`click`/`fill` but not `longpress`, and claiming it path-wide would -overstate coverage. The gate rejects `appliesTo` entries naming commands the -path does not dispatch, and rejects redundant full-coverage lists. +When a guarantee applies to only some path commands, cells may use command-scoped `appliesTo`. +For example, `verifyEvidence` applies to `press`/`click`/`fill`, but not `longpress`. +A path-wide claim would overstate coverage. +The gate rejects commands outside the path and redundant lists that cover all path commands. + +`responseConstruction` and `responseIdentity` remain separate. +A shared construction site and available identity fields need different completion strategies. +The former requires one Layer-2 refactor. +The latter requires per-path capability work. +`errorTaxonomy` is expected to split similarly later: + +- Stable codes and fallback classification. +- Rich selector diagnostics and hints. -`responseConstruction` and `responseIdentity` are deliberately separate: "use -one shared construction site" and "which identity fields this path can -provide" have different closure strategies (the former is a single Layer-2 -refactor; the latter is per-path capability work). `errorTaxonomy` is expected -to split the same way later — stable codes/fallback classification vs rich -selector diagnostics and hints — because direct runner paths can close codes -long before full diagnostics. +Direct runner paths can implement codes before full diagnostics. ```ts @@ -109,171 +115,167 @@ export const INTERACTION_DISPATCH_PATHS: Record< > = { /* every path, every cell */ }; ``` -Because `guarantees` is a `Record` over the guarantee union, **tsc fails** the -moment someone adds a guarantee without classifying it for every path, or adds -a path without classifying every guarantee. A unit gate -(`interaction-guarantees.test.ts`) additionally checks that: +`guarantees` is a `Record` over the guarantee union. +Thus, **tsc fails** if a new guarantee lacks classification for every path or a new path lacks any guarantee classification. +The unit gate `interaction-guarantees.test.ts` also verifies: -- every `via` string resolves to a real exported symbol (declarations cannot - rot into fiction); -- every `parityTable` names an existing fixture file; -- every `waived` reason is non-empty, and every `gap:` waiver carries a - `trackingIssue` — waivers must be owned, not merely visible. One umbrella - tracking issue with sub-issues split off as work is scheduled is - sufficient; the gate enforces the link, not the granularity. +- Every `via` string resolves to a real exported symbol. +- Every `parityTable` names an existing fixture file. +- Every `waived` reason is non-empty. +- Every `gap:` waiver has a `trackingIssue`. -This is the same "make the gap declare itself" pattern already proven in this -repo by `scripts/integration-progress-model.ts` (which caught the unclassified -`--verify` flag on #1064) and the cross-command apple-leak guard from the -platform consolidation (ADR 0009). +Waivers require ownership as well as visibility. +One umbrella issue is sufficient, with sub-issues added when work is scheduled. +The gate enforces the link, not issue granularity. + +The repository already uses this explicit-gap pattern in `scripts/integration-progress-model.ts` and ADR 0009's cross-command apple-leak guard. +The progress model detected the unclassified `--verify` flag on #1064. ### Layer 2 — Share: one implementation per rule, on both sides of the wire -Each guarantee has exactly one home that all TS paths import (most already -exist after #1075: `isNodeVisibleOnScreen`, `accumulateDisambiguationCandidate`, -`isSnapshotNodeInteractionBlocked`, `describeResolvedNode`, -`interactionResultExtra`, `reconcileNonHittableHintWithEvidence`). The -registry's `via` fields point at them; fallow's duplication gate keeps -re-implementations from creeping back. - -Rules that must run **runner-side** (the direct iOS path cannot see the daemon -tree) get pure-function Swift twins operating on plain geometry — no -`XCUIElement` — e.g. `TapPointPolicy.isAllowed(elementFrame:windowFrame:)`. -Parity is enforced by **golden fixture tables**: JSON files under -`contracts/fixtures/` consumed by three test suites — - -1. vitest asserts the TS rule over the table; -2. the runner's Swift unit tests (already compiled in CI by "Swift Runner Unit - Compile") assert the Swift twin over the same table; -3. the provider harness's fake runner derives its behavior from the same - table, so integration scenarios exercise the real contract rather than a - hand-written approximation. - -Drift between TS and Swift then turns CI red on whichever side changed, -without needing a simulator. - -`contracts/fixtures/dispatch-disclosure.json` is the table for -`AppErrorDetails.dispatched` on interaction failures: one row per producer -event, each with the value it must leave. The field has two values: `no` (the -operation never reached the device, or the command is a read, so a resend is -safe) and `unknown` (it may have landed, so observe before resending). Only a -producer that refuses before dispatch (target resolution, admission, a runner -pre-send refusal) may say `no`; no producer can prove execution on its failure -path, so there is no third value. - -Each request owns one dispatch ledger. Every bound runtime operation declares -its effect once, in `RUNTIME_OPERATION_EFFECTS`, a record over the runtime -operation keys, so an operation cannot bind without one. The request binding -records each `mutates` operation in the request's ledger when its send -returns, so no bound runtime operation sends a mutation outside the ledger. -Android ANR and blocking-dialog recovery is outside the ledger: its -`AndroidObservationAdapter.tap` and `openApp` calls reach the device without a -record. A nested request (a -batch step, a replay action, a delegated `find` click or fill) records into a -ledger of its own and moves its sends into its parent's. - -The request router discloses around every routed command. Once the ledger -holds a sent mutation, a failure is `unknown` with the sent count in -`details.dispatchedSteps`, whatever produced it and whatever the command -declares: a wrong `no` makes a consumer resend an action that already ran, -while a wrong `unknown` only costs an observation. Before that, the registry's -`recordingEffect` decides. For `'mutates-app'` the router keeps a producer's -value and fills `unknown` for a failure no producer classified. For -`'observes-app'` it sets `no` over any producer value. A command with no -declared effect passes through. - -`no` for a read means that a resend repeats no app-visible action. The -registry declares `record`, `trace`, and `perf` as `'observes-app'`, although -their recorder and profiler controls reach the device: no registry trait -separates a pure read from a device control, and those controls are declared -`repeatable` at the operation level, because the device refuses or ignores a -repeat. A producer that runs several device inputs inside one bound operation -(an Android double tap, an iOS press series) counts them in -`details.dispatchedSteps` itself, because the ledger sees one operation. - -Each row names its driver file by id prefix, and that file drives the real -producer. - -The Apple runner does not resend a mutating command whose first send may have -run: a restart resends only a command the first attempt provably did not write, -or a read-only one. A mutation whose reply stays lost (the runner dies -mid-command, or status recovery finds neither a retained result nor a runner -answer) fails with `reason: runner_reply_lost` and `dispatched: unknown` (rows -`ios-runner.transport.written-then-lost`, -`ios-runner.status.completed-without-retained-reply`, -`ios-runner.status.accepted`, `ios-runner.status.started`, -`ios-runner.status.notAccepted`, `ios-runner.status.probe-failed`, and -`ios-runner.status.unavailable`). The `ios-runner.status.failed*` rows carry the -runner's own answer instead. A read keeps its transport error and is resent. - -Remaining gaps: a failure before the router's locked scope (session -resolution, lock acquisition, lease and daemon-policy admission) never reaches -the disclosure and carries no `dispatched`. It sends nothing, but a consumer -must read the absent field as `unknown`. - -`runBatch` reports `no` only when every executed step is a command declared -`observes-app` (a read); any other executed step, including one with no -declared `recordingEffect`, makes the batch failure `unknown` for every caller, -including one that supplies its own `invoke`. - -For `responseFields`, one `buildInteractionResponseData(...)` becomes the only -construction site for interaction response payloads (this deletes the class of -bug where `fill @ref` rebuilt its response by hand and dropped `evidence`). A -small guard test — repo-precedented — fails if an interaction handler contains -a hand-rolled `responseData = {` literal. +Each guarantee has exactly one owner imported by every TypeScript path. +Existing owners include: + +- `isNodeVisibleOnScreen`. +- `accumulateDisambiguationCandidate`. +- `isSnapshotNodeInteractionBlocked`. +- `describeResolvedNode`. +- `interactionResultExtra`. +- `reconcileNonHittableHintWithEvidence`. + +The registry's `via` fields identify them. +Fallow's duplication gate prevents new duplicate implementations. + +Rules that must execute runner-side use pure-function Swift equivalents over plain geometry, without `XCUIElement`. +The direct iOS path cannot see the daemon tree. +`TapPointPolicy.isAllowed(elementFrame:windowFrame:)` is one example. +**Golden fixture tables** under `contracts/fixtures/` enforce parity through three consumers: + +1. Vitest asserts the TypeScript rule against the table. +2. Runner Swift unit tests assert the Swift equivalent against the same table. + CI already compiles them through "Swift Runner Unit Compile". +3. The provider harness's fake runner derives its behavior from the table. + Integration scenarios therefore exercise the real contract instead of a manual approximation. + +A TypeScript or Swift difference fails CI on the changed side without a simulator. + +`contracts/fixtures/dispatch-disclosure.json` defines `AppErrorDetails.dispatched` on interaction failures. +Each producer event has one row with its required value: + +- `no`: the operation never reached the device, or the command is a read, so a resend is safe. +- `unknown`: the operation may have reached the device, so observe before resend. + +Only pre-dispatch refusals from target resolution, admission, or runner pre-send checks may report `no` for an operation. +No producer can prove execution on its failure path, so no third value exists. + +Each request owns one dispatch ledger. +Every bound runtime operation declares its effect once in `RUNTIME_OPERATION_EFFECTS`, a record over runtime operation keys. +An operation cannot bind without an effect. +When a send returns, the request binding records each `mutates` operation in that request's ledger. +Thus, no bound runtime operation sends an unrecorded mutation. +Android ANR and blocking-dialog recovery remain outside the ledger. +Their `AndroidObservationAdapter.tap` and `openApp` calls reach the device without a record. +A nested request has its own ledger and transfers its sends to its parent's ledger. +Examples include batch steps, replay actions, and delegated `find` clicks or fills. + +The request router adds disclosure around every routed command. +After a sent mutation enters the ledger, any failure reports `unknown` and the sent count in `details.dispatchedSteps`. +This overrides both producer output and command declarations. +An incorrect `no` could cause a consumer to repeat an action that ran. +An incorrect `unknown` costs only an observation. +Before any sent mutation, registry `recordingEffect` controls disclosure: + +- For `'mutates-app'`, preserve the producer's value. + If no producer classified the failure, supply `unknown`. +- For `'observes-app'`, set `no` over any producer value. +- Without a declared effect, preserve the response unchanged. + +For reads, `no` means a resend repeats no app-visible action. +The registry declares `record`, `trace`, and `perf` as `'observes-app'`, although recorder/profiler controls reach the device. +No registry trait separates pure reads from device controls. +Those operations are `repeatable` because the device refuses or ignores a repeat. +If one bound operation performs several inputs, its producer counts them in `details.dispatchedSteps` because the ledger sees only one operation. +Examples include an Android double tap and an iOS press series. + +Each fixture row's id prefix identifies its driver file. +That file exercises the real producer. + +The Apple runner does not resend a mutation whose first send may have run. +After restart, it resends only a read-only command or a command that the first attempt provably did not write. +If a mutation's reply remains lost, failure carries `reason: runner_reply_lost` and `dispatched: unknown`. +This applies if the runner dies mid-command or status recovery finds neither a retained result nor a runner answer. +The fixture rows are: + +- `ios-runner.transport.written-then-lost`. +- `ios-runner.status.completed-without-retained-reply`. +- `ios-runner.status.accepted`. +- `ios-runner.status.started`. +- `ios-runner.status.notAccepted`. +- `ios-runner.status.probe-failed`. +- `ios-runner.status.unavailable`. + +The `ios-runner.status.failed*` rows instead carry the runner's own answer. +A read retains its transport error and is resent. + +An evidence gap remains before the router's locked scope. +Failures in session resolution, lock acquisition, lease admission, or daemon-policy admission never reach disclosure and carry no `dispatched`. +They send nothing, but consumers must interpret absence as `unknown`. + +`runBatch` reports `no` only if every executed step declares `observes-app`. +Any other executed step makes the batch failure `unknown`, including steps without `recordingEffect`. +This applies to every caller, including callers with their own `invoke`. + +For `responseFields`, `buildInteractionResponseData(...)` becomes the only construction site for interaction response payloads. +This prevents manual branches such as `fill @ref` from omitting `evidence`. +A small guard test follows repository precedent. +It fails if an interaction handler contains a manual `responseData = {` literal. ### Layer 3 — Prove: a contract suite generated from the registry -`test/integration/interaction-contract/` holds table-driven scenarios: fixture -tree × command × forced path. The fixture trees are the real shapes that found -this week's bugs, kept permanently: +`test/integration/interaction-contract/` contains table-driven scenarios: fixture tree × command × forced path. +Permanent fixture trees preserve the real failure shapes: + +- Closed drawer with all candidates offscreen → `offscreen_selector`/`offscreen_ref`. +- Stale or unknown `@ref` → `ref_not_found` with the stale-ref hint. +- Listed `@ref` or selector target without a usable center → `target_bounds_invalid`. + Bounds can be missing, non-finite, or negative. +- Listed `@ref` without a label for wait or scope → `ref_unlabeled`. +- Drawer item and visible twin → visible candidate wins despite on/offscreen ambiguity. +- Container with 0.07 px viewport overlap and offscreen center → refusal. +- Covered node → occlusion refusal. +- Non-hittable target with or without `--verify` → hint present or suppressed by evidence. +- Stripped-root tree without Application/Window → safe-default visibility. + +Tests force paths explicitly with `AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`. +This test-only control keeps cases stable when path selection heuristics change. +The gate traverses the Layer-1 registry. +It fails if a non-waived cell has no tagged contract case. +Thus, construction enforces matrix coverage without reviewer memory. -- closed drawer (all candidates off-screen) → `offscreen_selector`/`offscreen_ref`; -- stale or unknown `@ref` (errorTaxonomy) → `ref_not_found` beside the stale-ref hint; a listed - `@ref` whose node has no usable centre (missing, non-finite, or negative bounds), or a selector - target without one → `target_bounds_invalid`; a listed `@ref` with no label to wait on or scope - by → `ref_unlabeled`; -- drawer item + visible twin (ambiguous on/off-screen) → visible candidate wins; -- edge-grazing container (0.07 px viewport overlap, center off-screen) → still refused; -- covered node → occlusion refusal; -- non-hittable target ± `--verify` → hint present / suppressed by evidence; -- stripped-root tree (no Application/Window) → safe-default visibility. +### Closing the gaps: a hybrid strategy, not one answer -Path forcing is explicit (`AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`, -test-only) so cases stay stable when path-selection heuristics change. +Different guarantees require different mechanisms to close acknowledged gaps: -The gate closes the loop: it walks the Layer-1 registry and fails when any -non-waived cell has no contract case tagged for it. Coverage of the matrix is -therefore by construction, not by reviewer memory. +- **Runner-side parity for low-cost geometry-local rules.** Direct iOS offscreen/tappable-frame rules use pure geometry proven by golden tables. + They preserve the fast path's latency. +- **Delegation-on-error for semantic and rich-runtime cases.** These include `ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, and non-hittable refusal. + The fast path fails at low cost. + The runtime path supplies disambiguation and full diagnostics only when required. + **Delegation-on-error does not prove success-path parity.** + XCTest may find one hittable candidate that runtime rules would refuse or rank differently. + Those cells retain `gap:` waivers until parity tables or contract scenarios also prove success-path behavior. +- **Shared runtime preflight for native-ref.** The daemon snapshot already contains the ref's node. + Before `tapTarget`/`fillTarget`, examine offscreen, occlusion, and non-hittable conditions against that node. + A backend fast path can silently succeed, so delegation-on-error would never run. -### Closing the gaps: a hybrid strategy, not one answer +### Timeout policy joins the descriptor registry -The acknowledged gaps close by different mechanisms depending on what the -guarantee needs: - -- **Runner-side parity for cheap geometry-local rules** (offscreen / - tappable-frame on the direct iOS path). These are pure frame math, provable - with golden tables, and keep the fast path fast. -- **Delegation-on-error for semantic and rich-runtime cases** - (`ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, non-hittable - refusal): the fast path fails cheaply, and the runtime path supplies - disambiguation and full diagnostics only when needed. **Delegation-on-error - is not success-path parity**: it cannot catch the case where XCTest finds - one hittable candidate that runtime rules would refuse or rank differently. - Those cells stay `gap:` waivers until parity tables or contract scenarios - prove the success path too. -- **A shared runtime preflight for the native-ref path**: the ref came from a - daemon snapshot, so the node is already in hand — check offscreen / - occlusion / non-hittable against it *before* calling - `tapTarget`/`fillTarget`. A backend fast path can silently "succeed", so - delegation-on-error would never trigger there. +Two manually maintained client lists caused the `wait` timeout defect: -### Timeout policy joins the descriptor registry +- `isExplicitTimeoutCommand` controlled request-envelope budgets. +- `shouldResetDaemonAfterRequestTimeout` controlled timeout daemon policy. -The `wait` timeout bug existed because request-envelope budgets and -on-timeout daemon policy lived in two hand-maintained lists in the client -(`isExplicitTimeoutCommand`, `shouldResetDaemonAfterRequestTimeout`). Both are -replaced by declarations on the command descriptors (ADR 0008 registry): +Command descriptors in ADR 0008 replace both lists with declarations: ```ts timeoutPolicy: { @@ -282,155 +284,163 @@ timeoutPolicy: { } ``` -with a completeness gate over all public commands. Read-only polling commands -declare `preserve-daemon`; the client derives the envelope from the declared -budget source instead of special-casing command names. Since #1105 the -capture-resolving interaction commands (`click`, `fill`, `longpress`, `press`, -`type`, `get`, `is`) also declare `preserve-daemon`: their dominant timeout -mode is the same blocked accessibility capture as `snapshot`, and resetting -the daemon there destroyed every healthy app session the daemon owned. +A completeness gate covers every public command. +Read-only polling commands declare `preserve-daemon`. +The client derives the envelope from the declared budget source instead of command-name exceptions. +Capture-resolving interactions also declare `preserve-daemon` (#1105): + +- `click`. +- `fill`. +- `longpress`. +- `press`. +- `type`. +- `get`. +- `is`. + +Like `snapshot`, their dominant timeout cause is blocked accessibility capture. +A daemon reset would destroy every healthy app session it owns. ## Consequences -- Adding a dispatch path or a guarantee becomes a *forced* whole-matrix - decision: tsc will not compile an unclassified cell, and the gate will not - pass an untested one. Silent erosion is structurally impossible; explicit - waivers remain possible but visible and linkable to issues. -- Fast paths keep their latency wins. Their divergence is priced and - documented instead of discovered on-device. -- The Swift/TS split stops being a parity blind spot: golden tables are the - single source of truth for cross-language rules, and the fake runner stops - being a second, drifting implementation. -- Initial cost is mostly classification honesty: the first registry will - contain `waived('gap: ...')` cells (e.g. the maestro non-hittable fallback - intentionally waives `offscreen`), which is the point — the debt becomes a - diffable list instead of folklore. -- The registry doubles as documentation input for help/skill output ("what - press guarantees"), which matters for small-model agents that only read the - contract, never the code. +- A new path or guarantee requires a whole-matrix decision. + tsc rejects unclassified cells, and the gate rejects untested cells. + Silent loss is structurally impossible. + Explicit waivers remain possible, visible, and linked to issues. +- Fast paths retain latency benefits with documented differences and costs. +- Golden tables become the single source of truth for cross-language rules. + The fake runner no longer provides a second implementation that can diverge. +- Initial classification includes `waived('gap: ...')` cells. + For example, the maestro non-hittable fallback intentionally waives `offscreen`. + The debt becomes a reviewable list instead of undocumented knowledge. +- The registry also supplies documentation input for help/skill output, such as the guarantees of press. + This matters for small-model agents that read contracts without code. ### 2026-08-07 amendment: mutating ambiguity fails fast -Element-14 realized the matrix's previously owned success-path gap: four exact -`Team Standup` matches in distinct accessibility subtrees reached ordinary -runtime selector resolution, whose visible/depth/area ranking silently chose a -different semantic target. Mutating selectors no longer use geometry to choose -among distinct subtrees. - -The replacement contract is structural. Multiple matches collapse only when -all matches form one ancestor–descendant chain and every member resolves to the -same actionable node. Otherwise the mutation fails with `AMBIGUOUS_MATCH`, a -bounded list of snapshot candidate lines, and a partial ref frame generation so -the caller can retry one listed candidate immediately. On the direct XCTest path -this applies to mutating dispatches only: they count all raw exact matches before -hittability can select a winner and delegate multiple matches to the runtime -classifier. Reads (`querySelector`, and so `get`/`is`/`wait`) keep the prior rule -— prefer the single hittable match, ambiguous only when hittable matches compete -— because a read has no side effect to guard and failing it closed would turn a -decorative duplicate into an error. Maestro's explicit expected-point / -non-hittable compatibility path remains intentionally separate. +Element-14 demonstrated the owned success-path gap with four exact `Team Standup` matches in distinct accessibility subtrees. +Ordinary runtime selector resolution silently selected a different semantic target through visible/depth/area ranks. +Mutating selectors no longer use geometry to select between distinct subtrees. + +The replacement contract is structural. +Multiple matches collapse only if all form one ancestor–descendant chain and every member resolves to the same actionable node. +Otherwise, mutation fails with `AMBIGUOUS_MATCH`. +It returns bounded snapshot candidate lines and a partial ref frame generation for immediate retry with one listed candidate. +On direct XCTest paths, this applies only to mutating dispatches. +They count all raw exact matches before hittability can select a candidate. +Multiple matches delegate to the runtime classifier. +Reads through `querySelector`, and thus the original `get`/`is`/`wait` paths, retain the prior rule: + +- Prefer the single hittable match. +- Report ambiguity only if hittable matches compete. + +Reads have no side effect to protect. +A closed failure would turn a decorative duplicate into an error. +The amendment below updates which paths still use `querySelector`. +Maestro's explicit expected-point/non-hittable compatibility path remains intentionally separate. ### 2026-09-11 amendment: one collapse for one control, at both doors -Two clauses of the amendment above went stale as the read paths moved, and the -structural rule never reached the read door. - -- `querySelector` no longer backs `get`/`is`/`wait`. Those reads resolve against a - capture, through the `readUnique`/`readAny`/`readText` rows of - `packages/selectors/src/selector-pipeline.ts`, so the runner's hittable-preference - rule governs only the direct XCTest paths that still query it: the direct-iOS - touch fast path and the off-screen target probe. -- A control reported through its own accessibility wrapper answers a selector - twice. A regular iOS snapshot omits unverified hittability, so the ladder that - relates a wrapper to its control cannot fire, and #2482 collapsed that chain for - mutating resolution only: `is visible` and `get attrs` refused the same screen - as ambiguous while `press` tapped it. The collapse now sits beside the - classification that asks for it (`resolveUnverifiedWrapperControl`) and applies - where a refusal was the answer: the uniqueness rows — `is `, - `get attrs`, `screenshot --crop-on` — resolve the control instead of reporting - no match. A row that resolves before any refusal is untouched by it, and a - wrapper chain always resolves before one: depth separates a wrapper from its - control, so `get text` ranks onto the control and `wait`/`is exists` answer from - the document-order head without asking which element was meant. A candidate set - the rule does not recognize as one control — a cell and the button inside it, or - matches in distinct subtrees — still refuses. -- Replay verifies a recorded target by resolving its recorded selector again under - the same row's refusal rules, so the screen dispatch had resolved read as an - identity mismatch on the step's first replay. Verification names the collapsed - control too, which is the node dispatch acted on and the node the recorded - identity carries. +Two earlier clauses became stale after read paths changed. +The structural rule also needed to apply to reads: + +- `querySelector` no longer serves `get`/`is`/`wait`. + These reads resolve a capture through `readUnique`/`readAny`/`readText` in `packages/selectors/src/selector-pipeline.ts`. + The runner's hittable-preference rule therefore governs only remaining direct XCTest queries: direct-iOS touch fast path and offscreen target probe. +- A control's accessibility wrapper can make one selector match twice. + Regular iOS snapshots omit unverified hittability, so the wrapper-to-control relation cannot use that evidence. + A mutation-only collapse (#2482) let `press` succeed while `is visible` and `get attrs` reported ambiguity on the same screen. + `resolveUnverifiedWrapperControl` now places collapse beside its classification. + Uniqueness rows that otherwise refuse resolve the control: + - `is `. + - `get attrs`. + - `screenshot --crop-on`. + Rows that resolve before refusal remain unchanged. + A wrapper chain always resolves before refusal on those rows. + Depth makes `get text` rank the control above its wrapper. + `wait`/`is exists` use the first document-order match without requiring unique identity. + Candidate sets outside the one-control rule still refuse. + Examples include a cell with its internal button and matches in distinct subtrees. +- Replay verifies recorded targets by resolving their selectors under the same row's refusal rules. + Without equivalent collapse, the first replay could report identity mismatch for a target dispatch had resolved. + Verification therefore names the collapsed control, matching both dispatch and recorded identity. ### Synthesized iOS gesture policy -Synthesized iOS gestures (`scroll`, synthesized coordinate `tap`, synthesized -`drag`, and synthesized `sequence` tap/drag steps) are intentionally not folded -into the element dispatch-path matrix above. They do not resolve selectors or -refs, so claiming the element-targeting guarantees would be misleading. Their -AX-health, keyboard-probe, frame-source, activation-preflight, and -XCTest-coordinate fallback rules stay runner-local in -`RunnerTests+SynthesizedGesturePolicy.swift`. That Swift policy is the source of -truth because the current table has only three behaviors: - -- coordinate synthesized tap, standalone, as a `sequence` step, or as the - in-app `back` top-leading tap, never probes keyboards and may use the - coordinate fallback; -- default iOS scroll probes keyboards only after AX is known healthy and must - not fall back to `XCUICoordinate`; -- synthesized one-contact `gesture` plans may still use the coordinate fallback - before AX health is known, but stop using it once a snapshot stamps AX - unavailable. - -The non-obvious parts are covered by gated XCTest policy tests instead of a -cross-language mirror. A future sibling registry should only be introduced once -gesture policy grows beyond this small runner-local table or needs host-side -tooling to consume it directly. Synthesized coordinate contexts use screenshot -dimensions as their only frame source; cheaper frame sources such as -`app.frame`, windows, accessibility frames, or native screen bounds are -intentionally excluded because they can diverge from full-screen screenshot -coordinates on affected simulators. - -Target-authored `gesture drag` is classified as the `target-drag` path above because it resolves two -elements and therefore inherits the element guarantees and ADR 0012 identity requirements for both. +Synthesized iOS gestures intentionally remain outside the element dispatch-path matrix: + +- `scroll`. +- Synthesized coordinate `tap`. +- Synthesized `drag`. +- Synthesized `sequence` tap/drag steps. + +They resolve neither selectors nor refs. +Element-targeting guarantee claims would therefore mislead. +`RunnerTests+SynthesizedGesturePolicy.swift` retains runner-local rules for AX health, keyboard probes, frame sources, activation preflight, and XCTest-coordinate fallback. +This Swift policy is authoritative because the current table has only three behaviors: + +- Synthesized coordinate taps never probe keyboards and may use coordinate fallback. + This includes standalone taps, `sequence` steps, and the in-app `back` top-leading tap. +- Default iOS scroll probes keyboards only after known AX health. + It must not fall back to `XCUICoordinate`. +- Synthesized one-contact `gesture` plans may use coordinate fallback before known AX health. + After a snapshot reports unavailable AX, they stop using it. + +Gated XCTest policy tests cover the less obvious cases instead of a cross-language duplicate. +A sibling registry should be introduced only under either condition: + +- Gesture policy grows beyond this small runner-local table. +- Host-side tools need to consume it directly. + +Synthesized coordinate contexts use screenshot dimensions as their only frame source. +They intentionally exclude cheaper sources such as: + +- `app.frame`. +- Windows. +- Accessibility frames. +- Native screen bounds. + +Those sources can differ from full-screen screenshot coordinates on affected simulators. + +Target-authored `gesture drag` uses the `target-drag` path because it resolves two elements. +It inherits element guarantees and ADR 0012 identity requirements for both endpoints. Coordinate-authored drag remains outside this matrix. -Two-contact pan/pinch/rotate/transform planning is now owned by the typed -gesture-plan contract in [ADR 0013](0013-unified-gesture-plans.md). It remains a -sibling of this element-targeting matrix: coordinate gestures do not acquire -selector/ref guarantees merely because their native executor uses XCTest. +[ADR 0013](0013-unified-gesture-plans.md) owns two-contact pan/pinch/rotate/transform plans. +That contract remains separate from this element-targeting matrix. +A coordinate gesture does not acquire selector/ref guarantees because its native executor uses XCTest. ## Alternatives considered -- **Delete the fast paths** (single resolution spine): unacceptable — the - direct iOS path saves a full snapshot round trip per interaction and the - ref fast path is the backbone of replay throughput. The problem is not that - fast paths exist; it is that their trade-offs were implicit. -- **Typestate/branded types** ("cannot construct a response without proof the - guards ran"): cannot reach across the wire into Swift, and the proof tokens - go viral through every signature for marginal gain over the - registry-plus-gate split. Types are used where they are strongest — - completeness of the declaration — and tests where they are strongest — - truth of the declaration. -- **More integration tests without the registry**: this is the status quo - plus effort. Without the matrix as code, nothing forces a new path to - acquire the existing suite, which is exactly how this week's bugs happened. +- **Delete fast paths and use one resolution path.** Unacceptable. + Direct iOS avoids a full snapshot round trip per interaction. + The ref fast path is central to replay throughput. + Their implicit compromises require explicit contracts. +- **Use typestate/branded proof types.** Proof that response guards ran cannot cross the wire into Swift. + Proof tokens would spread through every signature for little additional benefit over the registry and gate. + Types enforce declaration completeness, while tests prove declared behavior. +- **Add integration tests without a registry.** This retains the existing failure mode with more effort. + Without a coded matrix, no mechanism requires a new path to acquire the existing suite. ### Optional observation before an iOS coordinate tap -A coordinate tap must not depend on a preceding XCTest snapshot failure. Its -optional text-input lookup may establish a concrete identity for a later bare -`type`; an absent or unavailable lookup establishes no typing witness. A runner -snapshot penalty can skip this work, but is only a performance optimization. +A coordinate tap must not depend on a preceding XCTest snapshot failure. +Its optional text-input lookup may establish concrete identity for a later bare `type`. +An absent or unavailable lookup establishes no typing witness. +A runner snapshot penalty can skip this work, but only as a performance optimization. + +The lookup owns a thread-bound issue scope in the runner recorder and returns a typed result. +Any recorded issue discards partial candidates, including an issue otherwise handled by AX suppression. +The scope excludes gesture dispatch and required text-entry reads. +Those failures retain existing mutation-outcome rules. -The lookup owns a thread-bound issue scope in the runner recorder and returns a -typed result. Any recorded issue, including one otherwise handled by AX suppression, -discards partial candidates. The scope excludes gesture dispatch and required -text-entry reads. Those failures retain the existing mutation-outcome rules. +The iOS PR lane exercises: -The iOS PR lane exercises a fresh runner with an unavailable probe, a suppressed -AX issue with a matching candidate, healthy coordinate tap followed by typing, -and failures outside the optional observation scope. These tests must not seed a -snapshot penalty to make the first tap safe. +- A fresh runner with an unavailable probe. +- A suppressed AX issue with a matching candidate. +- A healthy coordinate tap followed by typing. +- Failures outside the optional observation scope. -The recorder consumes optional-read issues before forwarding to XCTest. XCTest's -expected-failure API must not own this scope: in a long-lived command test it can -complete the enclosing test even when the command response succeeds. +These tests must not seed a snapshot penalty to make the first tap safe. +Before the recorder forwards optional-read issues to XCTest, it consumes them. +XCTest's expected-failure API must not own this scope. +In a long-lived command test, it can complete the enclosing test even if the command response succeeds. From 8de1193a8c71825691f7ab9a743bee0df36be1ec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:05:24 +0200 Subject: [PATCH 18/55] docs(adr): remove completed ADR 0013 process history --- docs/adr/0013-unified-gesture-plans.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/adr/0013-unified-gesture-plans.md b/docs/adr/0013-unified-gesture-plans.md index 590521a7d2..378b529d51 100644 --- a/docs/adr/0013-unified-gesture-plans.md +++ b/docs/adr/0013-unified-gesture-plans.md @@ -4,6 +4,8 @@ Accepted +Completed process history is in git history. + ## Context Gesture intent was previously interpreted at several private boundaries: command aliases produced @@ -143,8 +145,8 @@ should compose transport/device bindings with the shared platform adapter rather the interaction runtime. That `.ad` codec is the script format's own syntax, not a compatibility shim, and is not scheduled -for removal (amended 2026-07, issue #1216). Its only remaining callers are the CLI argv parse and -the `.ad` line parse — both the current public syntax — so there is nothing left to migrate off. +for removal. Its callers are the CLI argv parse and +the `.ad` line parse — both the current public syntax. A structured `.ad` gesture payload was rejected: it would make recordings unreadable and ungreppable for no behavioral gain. From 2edd926300625e0398a7c4cb895b4f0ac9065855 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:05:24 +0200 Subject: [PATCH 19/55] docs(adr): write ADR 0013 in ASD-STE100 --- docs/adr/0013-unified-gesture-plans.md | 430 ++++++++++++++----------- 1 file changed, 234 insertions(+), 196 deletions(-) diff --git a/docs/adr/0013-unified-gesture-plans.md b/docs/adr/0013-unified-gesture-plans.md index 378b529d51..500dc544c9 100644 --- a/docs/adr/0013-unified-gesture-plans.md +++ b/docs/adr/0013-unified-gesture-plans.md @@ -6,215 +6,253 @@ Accepted Completed process history is in git history. +## Rules at a glance + +- Normalize public gesture input once in `packages/contracts/src/gesture-normalization.ts`. +- Plan canonical intent in `packages/contracts/src/gesture-plan.ts`. Keep contact topology separate from motion. +- Reject removed timed public forms with actionable `INVALID_ARGS` messages. +- Keep Maestro's `endpoint-hold` profile private. Public `pan` uses `timed-pan`. +- Every injected sample must fit the freshly resolved active-app interaction viewport. Never clamp or distort the plan. +- Android touch execution never falls back to `adb input swipe`. +- Keep the positional `.ad` codec and shared `PUBLIC_GESTURE_SYNTAX` arity table. +- Repeated swipe and pause schedules must fit within 60 seconds. +- Before target-authored drag, resolve and verify both endpoints. Admit it only on Android touch devices and iOS/iPadOS. +- iOS timing changes require independent app-observed live simulator evidence. + ## Context -Gesture intent was previously interpreted at several private boundaries: command aliases produced -positional strings, daemon dispatch reparsed them, the `Interactor` exposed parallel semantic -methods, and each platform derived its own two-contact geometry. That duplicated validation, -responses, and behavior behind the three public surfaces: CLI, Node.js, and MCP. +Several private boundaries previously interpreted gesture intent: + +- Command aliases produced positional strings. +- Daemon dispatch parsed them again. +- `Interactor` exposed parallel semantic methods. +- Each platform derived its own two-contact geometry. + +This duplicated validation, responses, and behavior behind CLI, Node.js, and MCP. -Two-contact geometry also differed by platform. Android used a fixed radius while Apple derived a -radius from the app frame. Neither path validated every planned point against the active -interaction viewport before injection. +Two-contact geometry also differed by platform. +Android used a fixed radius, while Apple derived radius from the app frame. +Neither path validated every planned point against the active interaction viewport before injection. -`scroll` remains a separate command because it owns viewport/edge traversal and content-state -verification rather than a single physical gesture. Its Android physical movement still lowers to -the Android planned-touch executor. +`scroll` remains separate because it owns viewport/edge traversal and content-state verification, not one physical gesture. +Its Android physical movement still uses the Android planned-touch executor. ## Decision -Public gesture inputs normalize once in `packages/contracts/src/gesture-normalization.ts`. This is the -explicit public compatibility boundary: canonical semantic intent is produced before entering the -runtime. Deprecated arguments that have been removed (timed `swipe`, timed `gesture fling`, -`gesture rotate` `velocity`) are rejected with actionable `INVALID_ARGS` messages rather than -silently reinterpreted. The private daemon wire remains free to carry compatibility-only hints -that do not appear on public surfaces; Maestro timed swipes use -`internal.gestureExecutionProfile: 'endpoint-hold'` to preserve iOS fast-swipe-then-hold behavior -while still routing through the canonical `pan` input. - -The runtime plans canonical intent in `packages/contracts/src/gesture-plan.ts`. Contact topology is separate -from motion: - -- one contact: pan, fling, or target-authored drag with a complete pointer trajectory and an explicit - execution profile; -- two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories. - -`swipe` without a duration remains public sugar for a fixed-duration fling. Timed public forms -(`swipe x1 y1 x2 y2 durationMs`, `gesture fling direction x y distance durationMs`, -`gesture swipe preset durationMs`, and `gesture rotate degrees x y velocity`) are rejected; callers -must use `gesture pan` for deliberate timed translation and `gesture rotate` without `velocity`. -Maestro-authored swipes normalize to the canonical `pan` input (origin, delta, durationMs) and carry -the `endpoint-hold` execution profile through a daemon-internal compatibility seam -(`internal.gestureExecutionProfile`). This preserves the iOS fast-swipe-then-hold behavior that -matches Maestro's XCTest driver, without exposing the profile on the public command surface. Maestro -materializes its 400 ms default when duration is omitted. A genuine public `pan` uses the `timed-pan` -profile, so compatibility-only paths retain their release behavior without becoming a new semantic -intent. Pinch fixes translation and rotation at zero; rotate fixes -translation at zero and scale at one; two-finger pan fixes scale at one and rotation at zero; -transform can apply all three -components atomically. Intent remains on the plan even when aliases share an executor. - -The planner owns deterministic multi-touch geometry. Contacts start at -90 degrees, except Android -pinch starts horizontally because a vertical pinch is captured by common vertical app scroll -containers before the pinch recognizer activates. The same explicit planning profile preserves the -proven frame-count convention for dense two-contact trajectories: Android rounds while Apple -truncates the duration/16 ms frame count. The Android transport lowerer uses that same Android -sampling profile for one-contact endpoint plans. These are planner inputs, not adapter-generated -trajectories. The larger of pinch's initial and final spans is 40% of the -viewport's shorter side, preserving the proven Apple pinch geometry; other two-contact intents use -25% to keep translation and rotation trajectories compact. The other span follows from the requested scale, and both must -satisfy a 48-point reliability floor. Combined transforms progress translation, scale, and rotation -together inside one uninterrupted two-contact sequence so recognizers observe every intent without an -adapter regenerating geometry. The planner does not clamp points, cache the viewport, or distort -the requested components. Every injected sample must fit the freshly resolved active-app -interaction viewport; otherwise the request fails before injection with -`GESTURE_TRAJECTORY_OUT_OF_BOUNDS` and actionable details. Span and angle remain internal because -no established automation use case justifies a public tuning surface. +Public gesture inputs normalize once in `packages/contracts/src/gesture-normalization.ts`. +This is the explicit public compatibility boundary. +It produces canonical semantic intent before runtime entry. +Removed deprecated arguments receive actionable `INVALID_ARGS` errors instead of silent reinterpretation: + +- Timed `swipe`. +- Timed `gesture fling`. +- `gesture rotate` `velocity`. + +The private daemon wire may still carry compatibility-only hints absent from public surfaces. +Maestro timed swipes use `internal.gestureExecutionProfile: 'endpoint-hold'` with canonical `pan` input. +This preserves iOS fast-swipe-then-hold behavior. + +The runtime plans canonical intent in `packages/contracts/src/gesture-plan.ts`. +Contact topology is separate from motion: + +- One contact: pan, fling, or target-authored drag with a complete pointer trajectory and explicit execution profile. +- Two contacts: pan, pinch, rotate, or transform with two complete synchronized trajectories. + +Public `swipe` without duration remains an alias for fixed-duration fling. +These timed public forms are rejected: + +- `swipe x1 y1 x2 y2 durationMs`. +- `gesture fling direction x y distance durationMs`. +- `gesture swipe preset durationMs`. +- `gesture rotate degrees x y velocity`. + +For deliberate timed translation, callers must use `gesture pan`. +For rotation, they must use `gesture rotate` without `velocity`. +Maestro swipes normalize to canonical `pan` with origin, delta, and durationMs. +They carry `endpoint-hold` through daemon-internal `internal.gestureExecutionProfile`. +This matches Maestro's XCTest driver without a public execution-profile option. +If duration is absent, Maestro supplies its 400 ms default. +A public `pan` uses `timed-pan`. +Thus, compatibility paths retain release behavior without a new semantic intent. + +The plan retains intent even when aliases share an executor: + +- Pinch fixes translation and rotation at zero. +- Rotate fixes translation at zero and scale at one. +- Two-finger pan fixes scale at one and rotation at zero. +- Transform can apply all three components atomically. + +The planner owns deterministic multi-touch geometry. +Contacts start at -90 degrees, except Android pinch starts horizontally. +Common vertical app scroll containers capture vertical Android pinches before the pinch recognizer activates. +An explicit planning profile also preserves dense two-contact frame-count conventions. +Android rounds the duration/16 ms frame count. +Apple truncates it. +The Android transport lowerer uses the same Android sampling profile for one-contact endpoint plans. +These profiles are planner inputs, not trajectories generated by adapters. + +For pinch, the larger initial/final span is 40% of the viewport's shorter side, which preserves proven Apple geometry. +Other two-contact intents use 25% for compact translation and rotation trajectories. +Requested scale determines the other span. +Both spans must satisfy a 48-point reliability floor. +Combined transforms progress translation, scale, and rotation together in one uninterrupted two-contact sequence. +Recognizers observe every intent without adapter-generated geometry. +The planner does not clamp points, cache the viewport, or distort requested components. +Every injected sample must fit the freshly resolved active-app interaction viewport. +Otherwise, the request fails before injection with `GESTURE_TRAJECTORY_OUT_OF_BOUNDS` and actionable details. +Span and angle remain internal because no established automation use case justifies public controls. Platform adapters consume the canonical plan: -- Android's `executeAndroidTouchPlan` adapter seam sends planned touch, including gesture plans plus - the physical movement for scroll and long-press, to provider-native touch injection when - available, otherwise to the bundled instrumentation helper. One-contact endpoint plans lower in - `packages/platform-android/src/touch-plan-lowering.ts` to approximately 16 ms transport samples before - either injection path. Controlled directional scrolls accelerate for one frame, then decelerate - through release within the requested duration, without an appended tail. Inertial scrolls and - general one-contact plans retain linear interpolation; two-contact plans retain their exact - planned samples. Easing reduces release momentum but does not guarantee an exact content offset. Transport samples are typed as - strictly denser than the canonical endpoint pair, so skipping that lowering is a type error at - the injection seams instead of a silently sparse gesture. A stationary long-press needs no - viewport on the helper path; the executor adds the paired provider-owned viewport only for - provider-native touch. Android touch execution never falls back to `adb input swipe`. Public - scroll durations below one 16 ms planner frame normalize to that physical minimum and report the - executed duration. Scroll evidence reports absolute injected coordinates against zero-origin - extents that include the viewport offset. Because Android permits only one instrumentation owner - of `UiAutomation`, snapshot capture, gesture viewport resolution, and planned-touch injection - share one bundled automation helper: a live persistent helper session executes touch commands - directly, and without one the same helper runs one-shot. Nothing stops the snapshot session - around gestures anymore (amended 2026-07, issue #1275; previously a separate one-shot - multi-touch helper forced a session stop/restart around every local gesture). -- iOS lowers one-contact endpoint-hold plans to the established fast-swipe synthesis profile. That - profile reaches the endpoint in 100 ms, then holds there for the planned duration before lifting, - matching Maestro's XCTest driver. One-contact plans are linear and therefore carry only their - start and end samples, with the authored duration between them. Two-contact plans convert every - planned point to native orientation and feed the exact arrays to the private XCTest event bridge. - macOS lowers a one-contact plan to its drag executor and tvOS lowers it to remote direction. Core - admission and the Apple adapter both consume the same shared multi-touch support policy; - multi-touch remains capability-gated to iOS simulators. - - iOS deliberately retains three one-contact motion schedules behind the shared private-XCTest event - bridge. Apple's public drag API models the phases independently as an - [initial hold, movement velocity, and destination hold](https://developer.apple.com/documentation/xcuiautomation/xcuicoordinate/press%28forduration%3Athendragto%3Awithvelocity%3Athenholdforduration%3A%29). - UIKit likewise reports the velocity at finger-up together with the scroll view's predicted - [resting content offset](https://developer.apple.com/documentation/uikit/uiscrollviewdelegate/scrollviewwillenddragging%28_%3Awithvelocity%3Atargetcontentoffset%3A%29), - then applies the scroll view's configured - [post-lift deceleration rate](https://developer.apple.com/documentation/uikit/uiscrollview/decelerationrate-swift.property). - The distribution of movement over time is therefore observable gesture input, not an adapter - implementation detail. - - The three schedules shape that velocity differently. `endpoint-hold` moves quickly for 100 ms and - becomes stationary before lift. `timed-pan` and target-authored drag submit the authored samples - unchanged, preserving piecewise-linear movement plus explicit source and destination holds. The - runner's fused `scroll` reuses `endpoint-hold` for inertial release; controlled release instead - expands the movement to evenly spaced samples, one per roughly 16 ms capped at 30, using cubic - ease-out `s(t) = 1 - (1 - t)³`. A linear segment has constant movement velocity through its - endpoint unless a destination hold follows it; cubic ease-out decelerates from the first movement - and has zero slope at lift, with a starting velocity three times its average. Identical endpoints - and total durations can consequently produce different recognizer and deceleration outcomes. - Neither a destination hold nor an analytically zero endpoint slope proves a controlled release by - itself: XCTest event sampling and app recognizer thresholds can still leave observable post-lift - motion, so live evidence must measure the resulting content offset after pointer-up. - - Live iOS characterization of an eased sampled schedule against the linear ones in - [issue #1586](https://github.com/callstack/agent-device/issues/1586) confirmed that distinction: - the schedules crossed the same fling-recognizer thresholds in the tested range but produced - materially different post-release ScrollView positions and long-duration recognition behavior. The - distinction is intentional policy at the Apple adapter boundary, not a second interpretation of a - `GesturePlan`; changes require live evidence for both recognizer activation and post-release - content movement. -- WebDriver lowers a supported plan to synchronized W3C pointer action sources. A one-contact - endpoint plan becomes pointer down, one timed W3C `pointerMove` from start to end, and pointer up; - the driver owns interpolation across that W3C tick. Multi-touch remains capability-gated until a - provider proves it. - -The `Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of a -method per semantic alias. The old scalar Apple and Android multi-touch executors and the -public-command alias-to-positionals-to-reparse route are deleted. `.ad` keeps its established -positional syntax through one named codec; CLI, Node.js, and MCP send structured input. Providers -should compose transport/device bindings with the shared platform adapter rather than reimplement -the interaction runtime. - -That `.ad` codec is the script format's own syntax, not a compatibility shim, and is not scheduled -for removal. Its callers are the CLI argv parse and -the `.ad` line parse — both the current public syntax. -A structured `.ad` gesture payload was rejected: it would make recordings unreadable and ungreppable -for no behavioral gain. - -Argument arity for both callers comes from one table (`PUBLIC_GESTURE_SYNTAX`), so a form removed -from the CLI is removed from `.ad` in the same edit. A `.ad` script that still carries a removed -positional fails when the script is parsed, before the replay executes any device action, with the -offending line and its rewrite. Arity is the only thing checked at parse time: `${VAR}` tokens -resolve after planning, and interpolation never splits a token, so the count is decidable while the -values are not. The removal process these inputs follow — announce, warn for one minor release, -publish the migration guide, prove the repository is clean, then remove the branch and its tests — -is documented in the public -[gesture migration guide](https://oss.callstack.com/agent-device/docs/migrating-gestures). - -Repeated coordinate swipes are bounded at the public command contract and daemon trust boundary. -Individual count and pause limits prevent pathological fields, while the combined planned gesture -and pause schedule must fit within 60 seconds so valid fields cannot compose into an unbounded -session lock. - -Public two-finger pan is additive: `pointerCount?: 1 | 2` on pan and CLI -`--pointer-count 2`; omission remains one contact. Responses share the canonical -`kind`, `durationMs`, `pointerCount`, `from`, and `to` fields, followed by backend evidence. -Recording/replay keeps its existing public command identity and session semantics. - -Target-authored drag is additive: `gesture drag ` accepts a selector or pinned -snapshot ref at each endpoint, resolves both endpoints before device injection, then lowers their center -points to one uninterrupted single-pointer plan. The plan holds at the source, moves over the authored -duration, optionally holds at the destination, and releases. Both endpoint-resolution disclosures and -portable selector chains are returned to the caller. Recordings replace session-local refs at both -endpoints with those selector chains and attach one `targets-v1` annotation containing independent source -and destination identity evidence. Replay verifies both identities before pointer-down and guards both -independent endpoint resolutions against the verified elements. Ref admission happens for both endpoints -before either is dispatched, and the usual mutation boundary expires the frame after the gesture. Because -this contract includes the source hold, timed movement, and destination hold phases, target-authored drag -is admitted only on Android touch devices and iOS/iPadOS, whose adapters preserve the full plan. - -ADR 0011's coordinate path remains unchanged: coordinate-authored gestures do not resolve selectors or -refs and therefore cannot claim element-targeting guarantees. Target-authored drag explicitly runs the -shared selector/ref resolution preflight before it enters the coordinate gesture executor; its endpoint -resolution and recording contracts are tested at that composition seam. +- **Android.** `executeAndroidTouchPlan` receives gesture plans and physical movement for scroll and long-press. + If provider-native touch injection is available, it uses that provider. + Otherwise, it uses the bundled instrumentation helper. + `packages/platform-android/src/touch-plan-lowering.ts` lowers one-contact endpoint plans to approximately 16 ms transport samples before either injection path. + Controlled directional scrolls accelerate for one frame, then decelerate through release within the requested duration. + They append no extra movement. + Inertial scrolls and general one-contact plans retain linear interpolation. + Two-contact plans retain their exact planned samples. + Easing reduces release momentum but does not guarantee an exact content offset. + Transport types require samples strictly denser than the canonical endpoint pair. + Thus, omitted lowering causes a type error at injection boundaries instead of silently sparse input. + A stationary long-press needs no viewport on the helper path. + Only provider-native touch adds the paired provider-owned viewport. + Android touch execution never falls back to `adb input swipe`. + Public scroll durations below one 16 ms planner frame normalize to that minimum and report the executed duration. + Scroll evidence reports absolute injected coordinates against zero-origin extents that include viewport offset. + Android permits only one instrumentation owner of `UiAutomation`. + Snapshot capture, gesture viewport resolution, and planned-touch injection therefore share one bundled automation helper. + A live persistent helper session executes touch commands directly. + Without one, the same helper runs one-shot. +- **Apple.** iOS lowers one-contact endpoint-hold plans to the established fast-swipe synthesis profile. + It reaches the endpoint in 100 ms, holds for the planned duration, then lifts, as Maestro's XCTest driver does. + One-contact plans are linear and carry only start/end samples with the authored duration between them. + Two-contact plans convert every planned point to native orientation and send the exact arrays to the private XCTest event bridge. + macOS lowers one-contact plans to its drag executor. + tvOS lowers them to remote direction. + Core admission and the Apple adapter consume the same shared multi-touch support policy. + Multi-touch remains capability-gated to iOS simulators. + + iOS intentionally retains three one-contact motion schedules behind the shared private-XCTest event bridge. + Apple's public drag API independently models [initial hold, movement velocity, and destination hold](https://developer.apple.com/documentation/xcuiautomation/xcuicoordinate/press%28forduration%3Athendragto%3Awithvelocity%3Athenholdforduration%3A%29). + UIKit reports finger-up velocity with the predicted [resting content offset](https://developer.apple.com/documentation/uikit/uiscrollviewdelegate/scrollviewwillenddragging%28_%3Awithvelocity%3Atargetcontentoffset%3A%29). + It then applies the configured [post-lift deceleration rate](https://developer.apple.com/documentation/uikit/uiscrollview/decelerationrate-swift.property). + Movement distribution over time is therefore observable gesture input, not merely adapter implementation. + + The schedules produce different velocity profiles: + + - `endpoint-hold` moves for 100 ms, then remains stationary before lift. + - `timed-pan` and target-authored drag submit authored samples unchanged. + They preserve piecewise-linear movement and explicit source/destination holds. + - Fused runner `scroll` reuses `endpoint-hold` for inertial release. + Controlled release expands movement into evenly spaced samples, approximately one per 16 ms, capped at 30. + It uses cubic ease-out `s(t) = 1 - (1 - t)³`. + + A linear segment retains constant velocity through its endpoint unless a destination hold follows. + Cubic ease-out decelerates from first movement and has zero slope at lift. + Its starting velocity is three times its average. + Thus, identical endpoints and total durations can produce different recognition and deceleration outcomes. + Neither destination hold nor analytically zero endpoint slope alone proves controlled release. + XCTest event sampling and app recognizer thresholds can still produce post-lift motion. + Live evidence must therefore measure content offset after pointer-up. + + Live iOS characterization in [issue #1586](https://github.com/callstack/agent-device/issues/1586) compared an eased sampled schedule with linear schedules. + In the tested range, they crossed the same fling-recognizer thresholds. + However, post-release ScrollView positions and long-duration recognition differed materially. + This distinction is intentional Apple adapter policy, not a second interpretation of `GesturePlan`. + Changes require live evidence for recognizer activation and post-release content movement. +- **WebDriver.** Supported plans become synchronized W3C pointer action sources. + One-contact endpoint plans produce pointer down, one timed W3C `pointerMove` from start to end, and pointer up. + The driver owns interpolation across that W3C tick. + Multi-touch remains capability-gated until a provider proves it. + +`Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of one method per semantic alias. +The old scalar Apple/Android multi-touch executors are deleted. +The public alias-to-positionals-to-reparse route is also deleted. +`.ad` retains its established positional syntax through one named codec. +CLI, Node.js, and MCP send structured input. +Providers should compose transport/device bindings with the shared platform adapter instead of duplicating the interaction runtime. + +The `.ad` codec owns script syntax. +It is not a compatibility shim and is not scheduled for removal. +Its callers are CLI argv parse and `.ad` line parse, both current public syntax. +A structured `.ad` gesture payload was rejected because it would reduce readability and text search without behavioral benefit. + +One table, `PUBLIC_GESTURE_SYNTAX`, defines argument arity for both callers. +Thus, removal of a CLI form removes the `.ad` form in the same edit. +If a script retains a removed positional, parsing fails before any replay device action. +The error names the offending line and its rewrite. +Only arity is checked at parse time. +`${VAR}` tokens resolve after planning, and interpolation never splits a token. +Thus, counts are known before values. +The public [gesture migration guide](https://oss.callstack.com/agent-device/docs/migrating-gestures) defines this removal process: + +1. Announce the removal. +2. Warn for one minor release. +3. Publish the migration guide. +4. Prove that the repository has no remaining use. +5. Remove the branch and its tests. + +The public command contract and daemon trust boundary bound repeated coordinate swipes. +Individual count and pause limits prevent pathological fields. +The combined planned gesture and pause schedule must fit within 60 seconds. +Thus, individually valid fields cannot create an unbounded session lock. + +Public two-finger pan is additive through `pointerCount?: 1 | 2` and CLI `--pointer-count 2`. +Omission remains one contact. +Responses share canonical `kind`, `durationMs`, `pointerCount`, `from`, and `to`, followed by backend evidence. +Recording/replay retains public command identity and session semantics. + +Target-authored drag is additive through `gesture drag `. +Each endpoint accepts a selector or pinned snapshot ref. +Both endpoints resolve before injection. +Their center points become one uninterrupted single-pointer plan: + +1. Hold at the source. +2. Move over the authored duration. +3. If requested, hold at the destination. +4. Release. + +The caller receives both endpoint-resolution disclosures and portable selector chains. +Recordings replace both session-local refs with those chains. +One `targets-v1` annotation carries independent source and destination identity evidence. +Before pointer-down, replay verifies both identities. +It guards each independent endpoint resolution against the verified element. +Ref admission occurs for both endpoints before either dispatch. +After the gesture, the usual mutation boundary expires the frame. +Only Android touch devices and iOS/iPadOS admit target-authored drag. +Their adapters preserve the complete source-hold, timed-movement, and destination-hold plan. + +ADR 0011's coordinate path remains unchanged. +Coordinate-authored gestures resolve neither selectors nor refs and cannot claim element-targeting guarantees. +Before coordinate gesture execution, target-authored drag runs shared selector/ref preflight. +Tests prove endpoint resolution and recording contracts at that composition boundary. ## Consequences -- CLI, Node.js, MCP, runtime, and platform adapters share one normalization and planning model. -- Adding an ergonomic gesture alias does not add a platform implementation. -- One-finger pan remains the default and explicit two-finger pan retains pan intent. -- The active viewport is resolved for each gesture, so rotation, keyboard, and window changes do - not use stale geometry. -- On bare ADB, Android scroll and long-press require the bundled automation helper (the snapshot - helper APK) and `UiAutomation`; helper installation or runtime failure is surfaced directly - rather than degrading to an approximate `adb input swipe`. -- Canonical one-contact plans contain only two samples: a 10-second pan is two shared-plan samples - instead of 626. Android expands that plan only at the transport boundary, while two-contact - plans remain cadence-bounded because their synchronized geometry is part of their contract. -- Unit tests cover canonical plan shape, Android lowering, helper/provider payloads, and WebDriver - action construction. They cannot prove timing or event delivery inside the private XCTest bridge, - so iOS timing changes require live simulator evidence from an independent app-observed - postcondition. Runner start/end uptime deltas can supplement that evidence, but request echoes or - internal timing alone do not prove that the app recognized the intended gesture. +- CLI, Node.js, MCP, runtime, and adapters share one normalization and planning model. +- A new ergonomic alias does not require a platform implementation. +- One-finger pan remains default, and explicit two-finger pan retains pan intent. +- Each gesture resolves its active viewport. + Rotation, keyboard, and window changes therefore do not use stale geometry. +- On bare ADB, Android scroll and long-press require the bundled automation helper, the snapshot helper APK, and `UiAutomation`. + Helper install or runtime failure returns directly instead of approximate `adb input swipe` fallback. +- Canonical one-contact plans contain only two samples. + A 10-second pan therefore has two shared-plan samples instead of 626. + Android expands them only at the transport boundary. + Two-contact plans remain cadence-bounded because synchronized geometry belongs to their contract. +- Unit tests cover plan shape, Android lowering, helper/provider payloads, and WebDriver actions. + They cannot prove timing or delivery inside the private XCTest bridge. + iOS timing changes therefore require live simulator evidence from an independent app-observed postcondition. + Runner start/end uptime deltas can supplement it. + Request echoes or internal timing alone do not prove recognition of the intended gesture. ## Alternatives Considered -- Keep positional aliases and share only geometry math: rejected because validation, response, and - routing would still have two implementations. -- Make platform gesture APIs the source of truth: rejected because their timing, geometry, and - recognizer behavior differ and cannot provide cross-platform semantics. -- Make swipe a public synonym for pan: rejected because battle-tested gesture vocabulary treats a - fling/swipe as a quick directional throw and pan as deliberate timed translation. -- Add `two-finger-pan`: rejected because pointer count is topology, not a new motion intent. -- Expose span/angle controls: rejected until a concrete automation use case needs them. -- Consolidate scroll command semantics: rejected because edge/content verification is distinct; - only its Android physical touch execution is shared. +- **Keep positional aliases and share only geometry.** Rejected because validation, response, and routes would retain two implementations. +- **Make platform gesture APIs authoritative.** Rejected because their timing, geometry, and recognition differ and cannot define cross-platform semantics. +- **Make swipe a public synonym for pan.** Rejected because established gesture vocabulary distinguishes quick directional fling/swipe from deliberate timed pan. +- **Add `two-finger-pan`.** Rejected because pointer count defines topology, not new motion intent. +- **Expose span/angle controls.** Rejected until a concrete automation use case requires them. +- **Consolidate scroll semantics.** Rejected because edge/content verification is distinct. + Only Android physical touch execution is shared. From f4cbaa939862efe2c4e46287d42aea9c1033195d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:06:41 +0200 Subject: [PATCH 20/55] docs(adr): remove completed ADR 0014 process history --- docs/adr/0014-session-ref-frame-lifetime.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/adr/0014-session-ref-frame-lifetime.md b/docs/adr/0014-session-ref-frame-lifetime.md index a8943fcf06..1f446fd89b 100644 --- a/docs/adr/0014-session-ref-frame-lifetime.md +++ b/docs/adr/0014-session-ref-frame-lifetime.md @@ -2,13 +2,11 @@ ## Status -Accepted; implemented and enforced on `main` across all platforms. Fresh live evidence exercised +Accepted. Enforcement applies across all platforms. Fresh live evidence exercised every production seam except Android blocking-dialog recovery, which shipped as a documented, accepted gap: no deterministic repro exists to raise the system dialog on demand, and the seam's transition and abort logic are pinned by fixture regressions -(`android-system-dialog-ref-frame.test.ts`, `interaction-touch-android-readiness.test.ts`). The -staged migration plan lives in this file's git history; PR #1241 was an earlier compatible -transitional fix, not this architecture migration. +(`android-system-dialog-ref-frame.test.ts`, `interaction-touch-android-readiness.test.ts`). Completed process history is in git history. ## Rules at a glance From 2c81e550edfb6cb354479071aeb96baf6d5d9d90 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:07:31 +0200 Subject: [PATCH 21/55] docs(adr): write ADR 0014 in ASD-STE100 --- docs/adr/0014-session-ref-frame-lifetime.md | 840 +++++++++++--------- 1 file changed, 446 insertions(+), 394 deletions(-) diff --git a/docs/adr/0014-session-ref-frame-lifetime.md b/docs/adr/0014-session-ref-frame-lifetime.md index 1f446fd89b..7802f0a5e4 100644 --- a/docs/adr/0014-session-ref-frame-lifetime.md +++ b/docs/adr/0014-session-ref-frame-lifetime.md @@ -2,179 +2,201 @@ ## Status -Accepted. Enforcement applies across all platforms. Fresh live evidence exercised -every production seam except Android blocking-dialog recovery, which shipped as a documented, -accepted gap: no deterministic repro exists to raise the system dialog on demand, and the seam's -transition and abort logic are pinned by fixture regressions -(`android-system-dialog-ref-frame.test.ts`, `interaction-touch-android-readiness.test.ts`). Completed process history is in git history. +Accepted. Enforcement applies across all platforms. + +Completed process history is in git history. + +Fresh live evidence exercised every production seam except Android blocking-dialog recovery. +That seam has an accepted evidence gap because no deterministic procedure can raise the system dialog on demand. +Fixture regressions fix the expected transition and abort behavior in `android-system-dialog-ref-frame.test.ts` and `interaction-touch-android-readiness.test.ts`. ## Rules at a glance -Normative summary; the binding contracts and edge cases are in [Decision](#decision), and the -ref-frame vocabulary is promoted into `CONTEXT.md`. - -- A session owns at most one **ref frame** — the authorization namespace for mutation refs (epoch - exposed as `refsGeneration`, immutable source tree, `active`/`expired` state, `all` or bounded - issuance scope) — kept separate from the latest operational observation (`session.snapshot`). - The frame is one value on `SessionState`, replaced whole by a transition and never edited in - place; its type is nominal (`#`-private fields), so no module outside - `packages/daemon/src/core/ref-frame.ts` can construct a frame, edit one, or derive one from an existing - frame. What the type cannot judge is a whole frame moved unchanged — clearing the field, or - assigning another session's frame — which the field-owner gate still covers. -- A complete snapshot activates an `all` frame; `find`, settled diffs, and replay divergence - screens activate a bounded partial frame that supersedes the prior one; internal read-only - captures never activate, reindex, or expire a frame. -- Every mutating leaf expires the frame at the side-effect seam — after all pre-action guards, - immediately before the device op — with no success-only rollback; a post-dispatch failure still - leaves it expired. -- Mutation admission requires an active frame whose epoch and issuance scope authorize the ref; - rejections carry ordered typed reasons `ref_frame_expired` -> `ref_generation_mismatch` -> - `plain_ref_requires_complete_frame` -> `ref_not_issued`. -- Read-only ref consumers stay fail-open with a structured staleness warning only while the frame - retains the ref's evidence; missing evidence fails rather than resolving by positional - coincidence against a newer observation. -- Every daemon command declares a `RefFrameEffect` (`preserve` | `may-invalidate` | `delegated`) in - its descriptor daemon facet; the registry classification is completeness-gated. +[Decision](#decision) contains the binding contracts and edge cases. +`CONTEXT.md` defines ref-frame vocabulary. + +- A session owns at most one **ref frame**, separate from its latest operational observation, `session.snapshot`. + The frame holds an epoch exposed as `refsGeneration`, immutable source tree, `active`/`expired` state, and `all` or bounded issuance scope. + It is one value on `SessionState`. + Transitions replace it whole and never edit it in place. + Nominal `#`-private fields prevent construction, edits, or derivation outside `packages/daemon/src/core/ref-frame.ts`. + The field-owner gate covers unchanged whole-frame moves, including field clearance and assignment from another session. +- A complete snapshot activates an `all` frame. + `find`, settled diffs, and replay divergence screens activate bounded partial frames that supersede the prior frame. + Internal read-only captures never activate, reindex, or expire a frame. +- Every mutating leaf expires the frame after pre-action guards, immediately before the device operation. + No success-only rollback occurs. + Post-dispatch failure leaves the frame expired. +- Mutation admission requires an active frame whose epoch and issuance scope authorize the ref. + Typed rejection reasons have this order: + 1. `ref_frame_expired`. + 2. `ref_generation_mismatch`. + 3. `plain_ref_requires_complete_frame`. + 4. `ref_not_issued`. +- Read-only refs remain fail-open with structured staleness warnings only while retained frame evidence can resolve them. + Missing evidence fails instead of resolving against a newer observation by positional coincidence. +- Every daemon command declares `RefFrameEffect` (`preserve` | `may-invalidate` | `delegated`) in its descriptor daemon facet. + A completeness gate enforces registry classification. ## Context -Snapshot refs such as `@e12` are positional names inside one accessibility-tree namespace. They are -cheap and effective within that namespace, but they do not identify an element across navigation, -layout changes, or another capture whose positional ordering differs. A generation pin such as -`@e12~s42` identifies the namespace that issued the ref; it does not make the element stable. +Snapshot refs such as `@e12` are positional names inside one accessibility-tree namespace. +They are inexpensive and effective within that namespace. +They do not identify an element across navigation, layout changes, or captures with different positional order. +A generation pin such as `@e12~s42` identifies the issuing namespace. +It does not stabilize the element. + +The session model combines two distinct concepts in `session.snapshot`: -The current session model conflates two different concepts in `session.snapshot`: +1. The latest operational observation for selector matches, verification, settled observations, Android freshness, overlays, and diagnostics. +2. The caller-authorized tree whose refs may authorize later mutations. -1. the latest operational observation used by selector matching, verification, settling, Android - freshness, overlays, and diagnostics; and -2. the caller-authorized tree whose refs may be used for a later mutation. +This combination causes failures in both directions. +A mutation can change the device without another capture, so old positional refs appear current. +An internal read-only capture can replace `session.snapshot` and invalidate refs despite no caller mutation. +Partial `find` results, settled diffs, and replay divergence screens can clear the coarse stale marker for a whole tree after exposing few refs. -That conflation creates failures in both directions. A mutation can change the device without -capturing another tree, leaving the old positional refs apparently current. Conversely, an internal -read-only capture can replace `session.snapshot` and invalidate refs even though the caller performed -no mutation. Partial outputs such as `find`, settled diffs, and replay divergence screens can then -clear the coarse stale marker for the whole replacement tree even though they exposed only a few -refs. +Issue [#1239](https://github.com/callstack/agent-device/issues/1239) and PR [#1241](https://github.com/callstack/agent-device/pull/1241) demonstrate the first failure on iOS. +`snapshot -> press @e1 -> press @e2` can dispatch both presses if the first press takes no internal capture. +The second ref uses pre-action coordinates even if the first press navigated. +A false successful tap on a new screen is worse than an explicit stale-ref failure. -Issue [#1239](https://github.com/callstack/agent-device/issues/1239) and PR -[#1241](https://github.com/callstack/agent-device/pull/1241) demonstrate the first failure on iOS: -`snapshot -> press @e1 -> press @e2` can dispatch both presses when the first press performs no -internal capture. The second ref is resolved against pre-action coordinates even if the first press -navigated. A false successful tap on the new screen is worse than a clear stale-ref failure. +The lifetime contract must meet these requirements: -The lifetime contract must be consistent across platforms and dispatch paths, preserve fast paths, -remain compatible with selector-based replay, and add no automatic capture or per-node output bytes. +- Be consistent across platforms and dispatch paths. +- Preserve fast paths. +- Remain compatible with selector-based replay. +- Add no automatic capture or per-node output bytes. ## Decision -The terms introduced below are now the implemented model; the ref-frame vocabulary is promoted into -`CONTEXT.md`. The superseded coarse `snapshotRefsStale` marker has been removed: -read-only ref staleness is now derived from frame state — a plain ref warns once the frame has expired, -and a read-only capture no longer marks refs stale because it does not expire the frame. +The terms below define the implemented model and appear in `CONTEXT.md`. +The coarse `snapshotRefsStale` marker is removed. +Read-only ref staleness derives from frame state. +A plain ref warns after frame expiry. +Read-only captures do not mark refs stale because they do not expire the frame. ### Ref frames are separate from operational observations -A session owns at most one **ref frame**. The frame is the authorization namespace for mutation refs -and contains: +A session owns at most one **ref frame**. +This authorization namespace for mutation refs contains: -- a seeded, monotonic **ref-frame epoch**, exposed through the existing `refsGeneration` field and - accepted through the existing `~s` input suffix; -- the immutable source tree needed to resolve and guard the refs it issued; -- a state of `active` or `expired`; and -- an issuance scope of `all` or a bounded set of issued ref bodies. +- A seeded monotonic **ref-frame epoch**, exposed as `refsGeneration` and accepted through `~s`. +- The immutable source tree required to resolve and guard issued refs. +- State `active` or `expired`. +- Issuance scope `all` or a bounded set of issued ref bodies. -Operational captures update the latest observation used by selector and diagnostic machinery. They -do not silently replace, reactivate, or reindex the authorized ref frame. The implementation may -share immutable tree objects between the observation and frame, but it must not make their lifetimes -the same by aliasing mutable session state. +Operational captures update the latest observation for selectors and diagnostics. +They do not silently replace, reactivate, or reindex the authorized ref frame. +The implementation may share immutable tree objects between observation and frame. +It must not combine their lifetimes through mutable session-state aliases. -Only one frame is retained. Activating a new frame supersedes the previous frame and advances the -epoch; historical generation-to-tree caches are not introduced. Complete and partial frames retain -the immutable source tree because interaction guards and replay identity can depend on context outside -the emitted subset, including ancestors, siblings, overlays, and the viewport. The issuance set, not -the evidence tree, bounds partial authority. The frame and latest observation share immutable capture -data when they originate from the same capture; neither transition deep-copies the tree. +Only one frame is retained. +A new active frame supersedes the previous frame and advances the epoch. +No historical generation-to-tree cache is introduced. +Complete and partial frames retain the immutable source tree. +Interaction guards and replay identity can depend on ancestors, siblings, overlays, and viewport context outside emitted nodes. +The issuance set bounds partial authority, not the evidence tree. +If frame and observation originate from the same capture, they share immutable capture data. +Neither transition deep-copies the tree. -The existing `snapshotGeneration` implementation evolves behind one ref-frame module; the -`snapshotRefsStale` marker it originally paired with has since been removed. The -public name `refsGeneration` and the `@e12~s42` grammar remain unchanged for wire compatibility. +The existing `snapshotGeneration` implementation evolves behind one ref-frame module. +Its former companion `snapshotRefsStale` is removed. +For wire compatibility, public `refsGeneration` and `@e12~s42` grammar remain unchanged. ### Frame transitions -| Event | Frame transition | Plain refs | Pinned refs | -| ----------------------------------------- | -------------------------------------- | -------------------------- | ------------------------------------------------ | -| Internal observation that returns no refs | None | Unchanged | Unchanged | -| Complete frame activation | New active epoch, scope `all` | Accepted from this frame | Accepted when epoch matches | -| Non-empty partial publication | New active epoch, scope = emitted refs | Rejected | Accepted only for an emitted ref at this epoch | -| First possible device-side effect | Advance once to `expired` | Rejected | Previous epoch rejected | -| Additional effects while already expired | Idempotent | Rejected | Rejected | -| Sparse, failed, or unusable capture | None | Unchanged | Unchanged | -| Session reopen | New random-seeded lifetime | Rejected until publication | Old lifetime rejected probabilistically as today | - -A complete activation normally accompanies a command result that exposes the complete ref namespace -for the stored frame, including an interactive or intentionally scoped snapshot. An intentionally -scoped snapshot replaces authorization with exactly its returned namespace; it neither authorizes -matching ref bodies outside that scope nor retains the old `all` scope. - -A frame-activating capture scoped by `@ref` must first admit that scope ref against the current active -frame. To preserve the existing repeated scoped-snapshot contract after scoped output reindexes refs, -the session retains one bounded **scope lineage** for consecutive read-only repetitions of that exact -scope. The lineage stores the admitted semantic scope (label/selector chain), not the old positional -tree or mutation authority. A repeated `snapshot -s @ref` may resolve through that lineage only when -the immediately preceding frame publication was the same scoped capture. The first device-side effect, -an unrelated frame publication, or another scope clears the lineage. An arbitrary stale or unissued -scope ref still fails closed; it cannot borrow lineage or authorize a different subtree. Empty, sparse, -failed, or unusable repeated captures preserve the last useful frame and its still-consecutive lineage. -Selector-scoped captures follow their normal selector rules. - -Snapshot diffs, digests that omit the namespace, `find`, settled diffs/tails, and replay divergence -screens are partial external publications. They activate a partial frame only when at least one ref -and its epoch are visible to the caller or orchestrated workflow. An empty partial result does not -supersede useful existing authority. - -Publishing a partial frame intentionally supersedes the prior frame. This avoids retaining an -unbounded set of generation-specific trees and makes its limited authority explicit: only refs named -in that partial response can be used for mutation, and only with their response-level epoch. - -An explicit, non-diff complete snapshot leaf may activate a frame inside replay or batch before the -outer orchestrator returns, even when the outer response level later elides or digests that tree. This -execution-local activation is a deliberate compatibility exception for deterministic -`snapshot -> @ref` scripts. It authorizes only predeclared later bare-ref steps; it does not publish -pins to the outer caller or MCP, make the outer response ref-issuing, or add general result -interpolation. Internally injected, cached, freshness, and recovery captures never receive this -exception. A nested non-empty partial publication may supersede scope, but batch and replay have no -dynamic generation binding with which a later step could consume it. +| Event | Frame transition | Plain refs | Pinned refs | +| --- | --- | --- | --- | +| Internal observation that returns no refs | None | Unchanged | Unchanged | +| Complete frame activation | New active epoch, scope `all` | Accepted from this frame | Accepted when epoch matches | +| Non-empty partial publication | New active epoch, scope = emitted refs | Rejected | Accepted only for an emitted ref at this epoch | +| First possible device-side effect | Advance once to `expired` | Rejected | Previous epoch rejected | +| Additional effects while already expired | Idempotent | Rejected | Rejected | +| Sparse, failed, or unusable capture | None | Unchanged | Unchanged | +| Session reopen | New random-seeded lifetime | Rejected until publication | Old lifetime rejected probabilistically as today | + +Complete activation normally accompanies a result that exposes the stored frame's complete ref namespace. +This includes interactive or intentionally scoped snapshots. +An intentionally scoped snapshot replaces authorization with exactly its returned namespace. +It neither authorizes matching ref bodies outside the scope nor retains the old `all` scope. + +Before a frame-activating capture uses an `@ref` scope, it must admit that ref against the active frame. +Scoped output reindexes refs. +To preserve repeated scoped snapshots, the session retains one bounded **scope lineage** for consecutive read-only repetitions of that exact scope. +The lineage stores the admitted semantic scope as label/selector chain. +It stores neither the old positional tree nor mutation authority. +Repeated `snapshot -s @ref` may use lineage only if the immediately preceding frame publication was the same scoped capture. +These events clear lineage: + +- The first device-side effect. +- Unrelated frame publication. +- Another scope. + +Arbitrary stale or unissued scope refs fail closed. +They cannot use lineage or authorize another subtree. +Empty, sparse, failed, or unusable repetitions preserve the last useful frame and still-consecutive lineage. +Selector-scoped captures retain normal selector rules. + +These outputs are partial external publications: + +- Snapshot diffs. +- Digests that omit the namespace. +- `find`. +- Settled diffs/tails. +- Replay divergence screens. + +They activate partial frames only if the caller or orchestrated workflow sees at least one ref and its epoch. +An empty partial result does not supersede useful authority. +Partial publication intentionally supersedes the prior frame. +This avoids unbounded generation-specific trees and makes authority explicit. +Only refs named in that response can authorize mutations, with the response-level epoch. + +An explicit non-diff complete snapshot leaf may activate a frame inside replay or batch before the outer orchestrator returns. +This remains true if the outer response later omits or digests the tree. +This execution-local exception preserves deterministic `snapshot -> @ref` scripts. +It authorizes only predeclared later bare-ref steps. +It does not: + +- Publish pins to the outer caller or MCP. +- Make the outer response ref-issuing. +- Add general result interpolation. + +Internally injected, cached, freshness, and recovery captures never receive this exception. +Nested non-empty partial publication may supersede scope. +However, batch and replay have no dynamic generation binding for a later step to consume it. ### Mutation admission and read behavior -A ref-targeting mutation is admitted only when all of the following hold: +A ref mutation is admitted only if all conditions hold: -1. the frame is active; -2. a pinned ref carries the current epoch, or a plain ref is used against a complete `all` frame; -3. the ref body belongs to the frame's issuance scope; and -4. normal target guards and replay target-binding verification pass. +1. The frame is active. +2. A pin names the current epoch, or a plain ref targets a complete `all` frame. +3. The ref body belongs to the issuance scope. +4. Normal target guards and replay target-binding verification pass. -The target is resolved against the authorized frame, never against a newer operational observation -by positional coincidence. Android freshness captures may verify current identity or provide -comparison evidence, but they must not redefine which node an already-admitted `@eN` authorizes. +Resolution uses the authorized frame, never positional coincidence in a newer operational observation. +Android freshness captures may verify current identity or provide comparison evidence. +They must not redefine the node authorized by an already-admitted `@eN`. -Read-only ref consumers remain fail-open with a structured staleness warning only when the retained -frame contains the evidence needed to resolve that ref. A stale read is observable and recoverable; -a stale mutation can act on the wrong element. Missing frame evidence fails rather than falling -through to a newer operational observation by positional coincidence. Reads never reactivate a frame -or convert an unissued ref into mutation authority. +Read-only ref consumers remain fail-open with structured staleness warnings only if retained frame evidence can resolve the ref. +A stale read is observable and recoverable. +A stale mutation can affect the wrong element. +Missing evidence fails instead of using positional coincidence in a newer observation. +Reads never reactivate frames or give unissued refs mutation authority. -Rejected mutations use the existing error system. Admission failures are evaluated in the order -below and have typed reasons so callers can distinguish "capture a complete snapshot" from "use the -emitted pinned ref": +Rejected mutations use the existing error system. +Admission evaluates typed reasons in this order: -- `ref_frame_expired` when a side effect invalidated the current frame; -- `ref_generation_mismatch` when a pin names another epoch; -- `plain_ref_requires_complete_frame` when a plain ref targets a partial frame; and -- `ref_not_issued` when a pinned ref is outside a partial frame's issuance set. +1. `ref_frame_expired`: a side effect invalidated the current frame. +2. `ref_generation_mismatch`: a pin names another epoch. +3. `plain_ref_requires_complete_frame`: a plain ref targets a partial frame. +4. `ref_not_issued`: a pin is outside a partial frame's issuance set. -Each failure includes `details.ref`, `details.reason`, `details.currentGeneration`, the frame scope, -and `details.mintedGeneration` when the input is pinned. For example: +These reasons distinguish a need for complete snapshot capture from a need for an emitted pinned ref. +Each failure includes `details.ref`, `details.reason`, `details.currentGeneration`, and frame scope. +Pinned inputs also include `details.mintedGeneration`. +For example: ```text code: COMMAND_FAILED @@ -186,56 +208,61 @@ details.mintedGeneration: 42 # present only when the input was pinned hint: Capture a fresh interactive snapshot or use a stable selector. ``` -The daemon cannot truthfully report `mintedGeneration` for a plain ref. Error messages must not claim -that a found ref was missing or lacked bounds when the actual failure is lifetime expiration. +The daemon cannot truthfully report `mintedGeneration` for a plain ref. +If lifetime expiry caused failure, error messages must not claim that an existing ref was missing or lacked bounds. ### Expiration occurs at the side-effect seam -Expiration occurs after syntax, capability, freshness, target resolution, replay identity, -occlusion, visibility, bounds, and other pre-action guards pass, but immediately before the first -operation that may change device-visible element identity. Persist the in-memory transition before -awaiting that operation. - -Failure before this seam preserves the frame. Once the seam is crossed, success, timeout, -cancellation, connection loss, and ambiguous platform errors all leave it expired because the device -may already have acted. Crossing the seam also clears scoped-snapshot lineage. There is no success-only -rollback. - -This rule applies to every execution shape: - -- runtime ref/selector/coordinate paths expire before `tap`, `fill`, `longPress`, text input, or - gesture execution; -- native web ref paths run their existing ref preflight first, then expire before `clickRef` or - `fillRef`; -- direct iOS selector paths consume no ref but expire any existing frame before the fused runner - resolve-and-act request; -- platform-generic, app lifecycle, keyboard, alert, settings, and provider actions expire at their - actual leaf side-effect seam, not at generic request admission; -- `open`/relaunch against an existing session expires that session's frame before the first - close/launch dispatch. A failure or timeout after dispatch leaves it expired; a successful `close` - deletes the session and therefore retains no frame to restore; -- a fallback or retry may proceed after an error only when the error proves no prior side effect; - ambiguous post-dispatch errors neither restore the frame nor authorize an automatic second action; -- Android blocking-dialog recovery is itself device-mutating. Its recovery tap/relaunch must cross - the same idempotent seam before acting, even when invoked from apparent readiness work. If recovery - mutates before a requested ref action dispatches, that ref action aborts with - `ref_frame_expired`; it cannot continue against the recovered UI. Selector and coordinate actions - may re-resolve and continue under their existing policies. - -The seam is deliberately inside leaf execution. Expiring at router entry would reject the ref needed -by the current command, invalidate on ordinary validation failures, and mishandle multiplexed and -orchestrating commands. - -Some platform operations fuse their final status/target check and mutation in one request, including -direct iOS selectors and current native alert or keyboard actions. Until a lower-level "will act" -callback exists, dispatching that fused request is the conservative seam. A later `not found`, -`already hidden`, timeout, or no-alert result is post-seam and does not restore the frame. A command -with a separate definitive read-only status request preserves the frame when that status proves no -mutation request was dispatched. +Expiry occurs after all pre-action guards pass, including: + +- Syntax. +- Capability. +- Freshness. +- Target resolution. +- Replay identity. +- Occlusion. +- Visibility. +- Bounds. + +It occurs immediately before the first operation that may change device-visible element identity. +Before awaiting that operation, persist the in-memory transition. + +Pre-seam failure preserves the frame. +After the seam, success, timeout, cancellation, connection loss, and ambiguous platform errors all leave it expired. +The device may already have acted. +The seam also clears scoped-snapshot lineage. +No success-only rollback occurs. + +This rule covers every execution shape: + +- Runtime ref/selector/coordinate paths expire before `tap`, `fill`, `longPress`, text input, or gesture execution. +- Native web ref paths perform ref preflight, then expire before `clickRef` or `fillRef`. +- Direct iOS selector paths consume no ref but expire existing frames before the fused runner resolve-and-act request. +- Platform-generic, app lifecycle, keyboard, alert, settings, and provider actions expire at their actual leaf side-effect seam. + They do not expire at generic admission. +- Existing-session `open`/relaunch expires that session's frame before first close/launch dispatch. + Post-dispatch failure or timeout leaves it expired. + Successful `close` deletes the session, with no frame to restore. +- After an error, fallback or retry may proceed only if the error proves no prior side effect. + Ambiguous post-dispatch errors neither restore the frame nor authorize an automatic second action. +- Android blocking-dialog recovery mutates the device. + Before recovery tap/relaunch, it must cross the same idempotent seam, even during apparent readiness work. + If recovery mutates before a requested ref action dispatches, that action aborts with `ref_frame_expired`. + It cannot continue against the recovered UI. + Selector and coordinate actions may re-resolve and continue under existing policies. + +The seam intentionally belongs inside leaf execution. +Router-entry expiry would reject the current command's ref, invalidate ordinary validation failures, and mishandle multiplexed or orchestrating commands. + +Some operations combine final status/target checks and mutation in one request. +Examples include direct iOS selectors and current native alert or keyboard actions. +Until a lower-level “will act” callback exists, fused-request dispatch is the conservative seam. +Later `not found`, `already hidden`, timeout, or no-alert results do not restore the frame. +A separate definitive read-only status request preserves the frame if it proves no mutation request was dispatched. ### Command effects are daemon-owned and completely classified -The command descriptor's daemon facet declares a request policy: +The command descriptor's daemon facet declares request policy: ```ts type RefFrameEffect = 'preserve' | 'may-invalidate' | 'delegated'; @@ -243,233 +270,258 @@ type RefFrameEffect = 'preserve' | 'may-invalidate' | 'delegated'; type DaemonRefFrameEffect = RefFrameEffect | ((request: DaemonRequest) => RefFrameEffect); ``` -The daemon registry exposes a named resolver. Every daemon command is classified, but the -classification is an honesty/completeness guard rather than the transition site. `may-invalidate` -means that some successful path can cross a device side effect; the leaf implementation still calls -the ref-frame module only when that path is selected. Request-sensitive commands use a focused -resolver rather than pretending all subcommands behave alike. - -The registry is the exhaustive source of truth. The following table is non-exhaustive guidance for -classifying representative actions; it must not become a second prose registry: - -| Effect | Commands/actions | -| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `may-invalidate` | press, click, fill, longpress, type, focus, scroll, swipe, gesture, back, home, `tv-remote`, orientation, open/relaunch, trigger/push delivery, settings changes, install/reinstall, React Native overlay dismissal, and lifecycle operations that can replace the visible surface | -| Conditional resolver | keyboard status preserves while dismiss/return/input invalidate; alert get/wait preserve while accept/dismiss invalidate; find reads preserve while click/fill/focus/type delegate to their leaf mutation | -| `delegated` | batch, replay, and test/suite orchestrators; each nested leaf owns its transition | -| `preserve` | snapshots and other observation, assertion, screenshot, recording, trace, logs, events, network inspection, performance, inventory, capability, lease, and transport-management operations unless a selected subaction directly manipulates the visible surface | - -Clipboard reads and writes preserve the frame because pasteboard state alone does not change element -identity. A later paste/type action is independently invalidating. - -Generic routing is not an exception to the policy. `back`, `home`, `orientation`, `scroll`, `tv-remote`, -and `app-switcher` all reach the generic daemon leaf and must cross the same transition there when -they act. `app-switcher` is currently projected to the daemon by a direct writer but, unlike its -generic-routed siblings, omits an explicit daemon descriptor facet and relies on the registry's -generic fallback. Migration adds that facet and its effect classification; it does not invent a -specialized route. The completeness gate covers every command projected to the daemon, including -generic fallbacks, so a missing facet cannot hide an unclassified mutation. Mutations performed by -unrelated external tools remain outside this session guarantee. - -This policy is not derived from the runner-side classification. Apple runner command traits govern -retry eligibility, launch policy, and the recorded-failure conversion at a lower wire-command seam, -while the TypeScript `readOnly` trait is itself consumed as several daemon decisions: read-only -resend, session-invalidation skip, transport error classification, and readiness preflight. -`refFrameEffect` governs daemon session authorization and includes commands that never reach the -Apple runner. Narrow consistency tests may cover direct mappings, but blanket parity would couple -different concepts. - -Frame admission and transitions are serialized by the existing per-session request lock. The frame -is shared session state, not per-client or per-lease history. Generation pins make the rejected epoch -and reason precise; they do not reserve a frame or prevent another caller's mutation from expiring it. +The daemon registry exposes a named resolver. +Every daemon command is classified. +Classification checks honesty/completeness, not transitions. +`may-invalidate` means some successful path can cross a device side effect. +Only after that path is selected does the leaf call the ref-frame module. +Request-sensitive commands use a focused resolver instead of one classification for all subcommands. + +The registry is the exhaustive source of truth. +This non-exhaustive table guides representative classifications. +It must not become a second prose registry: + +| Effect | Commands/actions | +| --- | --- | +| `may-invalidate` | press, click, fill, longpress, type, focus, scroll, swipe, gesture, back, home, `tv-remote`, orientation, open/relaunch, trigger/push delivery, settings changes, install/reinstall, React Native overlay dismissal, and lifecycle operations that can replace the visible surface | +| Conditional resolver | keyboard status preserves; dismiss/return/input invalidate. Alert get/wait preserve; accept/dismiss invalidate. Find reads preserve; click/fill/focus/type delegate to leaf mutation. | +| `delegated` | batch, replay, and test/suite orchestrators; each nested leaf owns its transition | +| `preserve` | snapshots and other observation, assertion, screenshot, recording, trace, logs, events, network inspection, performance, inventory, capability, lease, and transport-management operations, unless a selected subaction directly manipulates the visible surface | + +Clipboard reads and writes preserve the frame because pasteboard state alone does not change element identity. +A later paste/type action invalidates independently. + +Generic routes have no exception. +`back`, `home`, `orientation`, `scroll`, `tv-remote`, and `app-switcher` must cross the transition in the generic daemon leaf when they act. +A direct writer currently projects `app-switcher` to the daemon. +Unlike generic-routed siblings, it omits an explicit daemon descriptor facet and uses the registry's generic fallback. +The migration adds that facet and effect classification without a specialized route. +The completeness gate covers every daemon-projected command, including generic fallbacks. +Thus, a missing facet cannot hide an unclassified mutation. +Mutations from unrelated external tools remain outside this session guarantee. + +Runner-side classification does not define this policy. +Apple runner traits govern retry eligibility, launch policy, and recorded-failure conversion at a lower wire-command boundary. +The TypeScript `readOnly` trait controls: + +- Read-only resend. +- Session-invalidation skip. +- Transport error classification. +- Readiness preflight. + +`refFrameEffect` governs daemon session authorization, including commands outside the Apple runner. +Narrow consistency tests may cover direct mappings. +Blanket parity would combine distinct concepts. + +The existing per-session request lock serializes frame admission and transitions. +The frame is shared session state, not per-client or per-lease history. +Generation pins identify rejected epochs and reasons precisely. +They neither reserve frames nor prevent another caller's mutation from expiring them. No second ref-specific mutex is introduced. ### Batch, replay, recording, and repeated actions -Batch and replay invoke their steps sequentially through ordinary daemon dispatch and are -`delegated`. They never expire a frame merely because the outer command began. Frame state naturally -carries between leaf steps: - -- `snapshot -> press @e1 -> press @e2` executes the first press and rejects the second; -- `snapshot -> press @e1 -> press @e1 -> press @e2` also stops on the second press, before `@e2`; -- `snapshot -> press @e1 -> snapshot -> press @e2` is admitted; -- selector-only and coordinate-only batches retain their request-round-trip advantage; and -- a stale-ref failure stops batch through its existing policy and preserves `step`, `command`, - `executed`, `total`, `partialResults`, diagnostic fields, and `details.reason`. - -Batch does not gain nested MCP pin traversal, intermediate-result interpolation, an unsafe -"assume stable refs" override, or a second static lifecycle interpreter. Normal leaf dispatch is the -single enforcement path. - -Recorded `.ad` scripts remain selector-first. The recorder continues converting successfully -resolved refs to selector chains and refuses to persist an unresolved session-bound bare ref. -Generation suffixes remain runtime evidence and are stripped from durable recordings because seeded -epochs are not portable across sessions. Legacy hand-written `.ad` scripts that mutate through -several bare refs from one snapshot become intentionally incompatible; they must capture between -mutations or use selectors. Replay target-binding verification completes before the side-effect seam; -identity divergence preserves the frame, while an action failure after dispatch leaves it expired. - -`press --count`, double-tap, hold, jitter, `swipe --count`, and platform sequence chunks are one -logical command. They resolve or plan once, expire once before the first repetition, and execute the -original physical point/plan for every repetition. They do not re-resolve the target or reactivate the -frame between repetitions. Failure in any later repetition or chunk leaves the frame expired. This -preserves the explicit repeated-physical-interaction contract while preventing later commands from -reusing the frame. +Batch and replay invoke sequential steps through ordinary daemon dispatch and declare `delegated`. +Outer command start never expires a frame. +Frame state passes naturally between leaves: + +- `snapshot -> press @e1 -> press @e2` executes the first press and rejects the second. +- `snapshot -> press @e1 -> press @e1 -> press @e2` stops on the second press, before `@e2`. +- `snapshot -> press @e1 -> snapshot -> press @e2` is admitted. +- Selector-only and coordinate-only batches retain their request-round-trip advantage. +- Stale-ref failure stops batch under existing policy. + It preserves `step`, `command`, `executed`, `total`, `partialResults`, diagnostics, and `details.reason`. + +Batch gains no nested MCP pin traversal, intermediate-result interpolation, unsafe ref-stability override, or second static lifecycle interpreter. +Normal leaf dispatch is the sole enforcement path. + +Recorded `.ad` scripts remain selector-first. +The recorder converts successfully resolved refs to selector chains. +It refuses to persist unresolved session-bound bare refs. +Generation suffixes remain runtime evidence and are removed from durable recordings because seeded epochs are not portable across sessions. +Legacy handwritten `.ad` scripts that mutate several bare refs from one snapshot become intentionally incompatible. +They must capture between mutations or use selectors. +Replay target-binding verification finishes before the seam. +Identity divergence preserves the frame. +Post-dispatch action failure leaves it expired. + +These repetitions form one logical command: + +- `press --count`. +- Double-tap. +- Hold. +- Jitter. +- `swipe --count`. +- Platform sequence chunks. + +They resolve or plan once and expire once before the first repetition. +Every repetition executes the original physical point/plan. +They neither re-resolve the target nor reactivate the frame between repetitions. +Later repetition or chunk failure leaves the frame expired. +This preserves explicit repeated physical actions while preventing frame reuse by later commands. ### Partial issuance and MCP -`find`, settled diffs/tails, and replay divergence screens publish only the refs they actually return. -They must not call a whole-frame "refs issued" marker. - -Read-only `find` may partially publish its returned ref. A mutating `find` returns the acted ref only -as diagnostic pre-action identity: it must omit `refsGeneration`, must not publish or activate a -frame, and cannot make that ref valid after the nested action. - -The MCP executor continues storing per-ref pins by state-directory/session scope. Issuance handling -must distinguish the selected `find` action, not infer issuance from the command name alone. A -read-only `find` with `refsGeneration` merges its partial pin. A mutating `find` without -`refsGeneration` is explicitly non-issuing and leaves every remembered pin unchanged; it must not -enter a generic missing-generation branch that clears the session's pin scope. The executor must carry -enough normalized command/action context to make that distinction. Remembered pins are not cleared on -mutation: sending the remembered old epoch is how the daemon produces a precise stale rejection. MCP -currently neither traverses refs nested inside batch inputs nor learns refs from nested batch results; -this remains an explicit non-goal. - -The response-level `refsGeneration` field remains the token-economical authority. Snapshot nodes never -gain per-node generation bytes. Every reusable ref in a partial CLI text result must render in -ready-to-copy `@eN~s` form. JSON and Node.js responses retain plain ref bodies plus one -response-level generation; callers must pair them before a mutation. MCP may render a plain ref because -it stores and forwards the pin internally. Complete snapshot trees remain plain to protect the token -budget. A mutating `find` diagnostic ref is never rendered or stored as pinned because it carries no -`refsGeneration`. +`find`, settled diffs/tails, and replay divergence screens publish only returned refs. +They must not mark the whole frame as issued. + +Read-only `find` may partially publish its returned ref. +Mutating `find` returns its acted ref only as diagnostic pre-action identity. +It must omit `refsGeneration`. +It must not publish or activate a frame. +It cannot validate that ref after the nested action. + +The MCP executor retains per-ref pins by state-directory/session scope. +Issuance must distinguish the selected `find` action instead of command name alone. +Read-only `find` with `refsGeneration` merges its partial pin. +Mutating `find` without `refsGeneration` is explicitly non-issuing and leaves remembered pins unchanged. +It must not enter a generic missing-generation branch that clears the session's pin scope. +The executor must carry sufficient normalized command/action context for this distinction. +Mutation does not clear remembered pins. +The old epoch lets the daemon return a precise stale rejection. +MCP neither traverses nested batch input refs nor learns refs from nested batch results. +Both remain explicit non-goals. + +Response-level `refsGeneration` retains authority without repeated per-node bytes. +Snapshot nodes never gain generation bytes. +Every reusable partial CLI text ref must render as ready-to-copy `@eN~s`. +JSON and Node.js retain plain ref bodies with one response-level generation. +Before mutation, callers must pair them. +MCP may render plain refs because it stores and forwards pins internally. +Complete snapshot trees remain plain to preserve the token budget. +A mutating `find` diagnostic ref never renders or stores as pinned because it has no `refsGeneration`. ## Performance and compatibility -Expiration is O(1): one monotonic epoch transition and bounded in-memory state. It adds no capture, -platform request, daemon round trip, or per-node output. The model retains at most one authorized frame -and one latest operational observation, sharing immutable capture objects rather than deep copies; it -never caches an unbounded history of full trees. Existing capture size limits remain the bound. Memory -pressure must not silently expire or swap a frame because that would turn deterministic admission into -environment-dependent behavior. - -The real cost is workflow-level and intentional. A ref-oriented sequence that performs several -mutations may need an explicit observation between them. Callers can avoid unnecessary captures by -using selectors, direct selector fast paths, a settled partial result whose emitted ref is pinned, or -`--count` for a repeated physical action at one resolved target. The daemon never auto-captures merely -to renew refs. - -Older clients remain wire-parse-compatible: command grammar, `refsGeneration`, and `~s` remain -unchanged, and structured error details are additive, so ADR 0006 requires no protocol bump. They are -not fully behaviorally compatible. Unsafe plain-ref mutation chains newly fail. A fused command can -also expire the frame when its eventual result is a no-op, such as `keyboard dismiss` reporting that -the keyboard was already hidden or an alert action reporting no alert. Read-only refs that lack -retained frame evidence newly fail instead of warning and resolving by positional coincidence against -a newer observation. These changes require CLI/help, changelog, `.ad` migration, and batch guidance. - -It composes with existing ADRs as follows: - -- ADR 0003/0008: daemon-owned effect policy is composed in the descriptor daemon facet and projected - only through the daemon registry; -- ADR 0004: capture plans and sparse-quality rules are unchanged, and no capture is added; -- ADR 0005: runner liveness/preflight state remains independent from session ref authorization; -- ADR 0006/0010: no RPC bump; typed `details.reason` and actionable hints follow the error contract; -- ADR 0007: the frame is session-scoped under the existing session/request lock, not client- or - lease-local; -- ADR 0009: one target model applies to Apple, Android, Linux, and web while public platform output - remains projected; -- ADR 0011: lifetime is a sibling session-sequence invariant, not another path-by-element-guarantee - cell; every mutating path still crosses the shared seam; -- ADR 0012: durable target identity remains selector/evidence based, and divergence refs become - honest partial issuance; and +Expiry is O(1): one monotonic epoch transition and bounded in-memory state. +It adds no capture, platform request, daemon round trip, or per-node output. +At most one authorized frame and one latest operational observation remain retained. +They share immutable capture objects without deep copies. +The model never caches unbounded full-tree history. +Existing capture size limits remain the bound. +Memory pressure must not silently expire or replace frames. +Otherwise, deterministic admission would depend on the environment. + +The intentional cost affects workflows. +Several ref mutations may require explicit observations between them. +Callers can avoid unnecessary captures through: + +- Selectors. +- Direct selector fast paths. +- A pinned emitted ref from settled partial output. +- `--count` for repeated physical actions at one target. + +The daemon never auto-captures only to renew refs. + +Older clients remain wire-parse-compatible. +Command grammar, `refsGeneration`, and `~s` remain unchanged. +Structured error details are additive, so ADR 0006 requires no protocol increment. +Behavioral compatibility is incomplete: + +- Unsafe plain-ref mutation chains now fail. +- Fused commands can expire frames despite eventual no-op results. + Examples include already-hidden `keyboard dismiss` and alert actions with no alert. +- Read-only refs without retained evidence fail instead of warning and resolving by positional coincidence in newer observations. + +These changes require CLI/help, changelog, `.ad` migration, and batch guidance. + +This contract composes with existing ADRs: + +- ADR 0003/0008: daemon-owned effect policy composes in the descriptor daemon facet and projects only through the daemon registry. +- ADR 0004: capture plans and sparse-quality rules remain unchanged, with no added capture. +- ADR 0005: runner liveness/preflight remains independent of ref authorization. +- ADR 0006/0010: no RPC increment; typed `details.reason` and actionable hints follow the error contract. +- ADR 0007: frames use the existing session/request lock, outside client/lease-local state. +- ADR 0009: Apple, Android, Linux, and web share one target model with projected public platform output. +- ADR 0011: lifetime spans session sequences instead of another element-guarantee cell. + Every mutating path still crosses the shared seam. +- ADR 0012: durable target identity retains selectors/evidence; divergence refs issue bounded partial authority. - ADR 0013: coordinate gestures expire frames but remain outside the element guarantee matrix. ## Consequences -- A stale ref cannot silently mutate a different element after navigation; the caller receives a - typed failure and must choose a fresh observation or a selector. -- Ref-oriented workflows that perform multiple logical mutations must capture again between them, - consume an honestly issued settled ref, use selectors, or use one repeated-action command such as - `--count`. Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must - change. -- Conservative fused requests may consume ref authority even when the platform later reports that no - action was needed. This favors avoiding a wrong-screen action over preserving refs after an - ambiguous dispatch. -- A read without retained evidence fails rather than resolving the same positional ref body in a - newer tree. Reads with retained evidence keep the structured warning behavior. -- Consecutive repeated scoped snapshots retain their admitted semantic scope, while any mutation, - unrelated publication, or different scope breaks that lineage. -- The implementation pays bounded session-memory bookkeeping but adds no automatic capture, platform - call, round trip, or per-node response bytes. +- Stale refs cannot silently mutate a different element after navigation. + Typed failure requires a fresh observation or selector. +- Multiple logical ref mutations require captures between them, issued settled refs, selectors, or one repeated-action command such as `--count`. + Legacy handwritten `.ad` scripts that reuse bare refs from one snapshot must change. +- Conservative fused requests may consume authority even if the platform reports no required action. + This prioritizes prevention of wrong-screen actions over ref preservation after ambiguous dispatch. +- Reads without retained evidence fail instead of using positional coincidence in newer trees. + Reads with retained evidence preserve structured warnings. +- Consecutive scoped snapshots retain admitted semantic scope. + Mutation, unrelated publication, or another scope clears lineage. +- Bounded session-memory bookkeeping adds no automatic capture, platform call, round trip, or per-node response bytes. ## Required evidence -The implementation is not complete until contract and provider tests prove: - -1. a bare and pinned second mutation reject after an unobserved first mutation; -2. definitive pre-seam validation, target, and replay-identity failures preserve the frame; -3. timeout, cancellation, connection loss, and other post-seam failures leave it expired; -4. runtime ref/selector/coordinate, native ref, direct selector, provider-backed interaction, and - provider-backed lifecycle/generic paths cross the same seam; -5. Android freshness cannot retarget an admitted ref through positional coincidence; -6. read-only operational captures do not replace the authorized frame; -7. partial publication admits only emitted pinned refs, reports each typed admission failure, cannot - bless unrelated plain refs, and CLI text renders every reusable partial ref with its pin while JSON - and Node.js retain the response-level representation; -8. complete publication admits its plain namespace while sparse/unusable captures do not; -9. mutating `find` omits `refsGeneration` and never pins or activates its pre-action ref as current - post-action state; an MCP sequence that first remembers a snapshot pin, then performs mutating - `find`, still forwards the remembered old pin on a later ref input so the daemon rejects it as stale; -10. batch and replay delegate per step, retain their existing failure/partial-result contracts, and - allow `snapshot -> @ref` with an intermediate digest without exposing pins or dynamic bindings; -11. recorded scripts contain selectors rather than portable generation claims; -12. press/swipe series and Apple chunks resolve and expire exactly once; -13. Android dialog recovery expires before its first recovery side effect and aborts an outstanding - ref action after recovery mutates; -14. consecutive repeated `snapshot -s @ref` calls resolve the same admitted semantic scope after - scoped output reindexes refs; a mutation, unrelated publication, or different scope breaks the - lineage; an arbitrary stale ref cannot use it; and empty/unusable repetition preserves the last - useful frame and still-consecutive lineage; -15. fused keyboard/alert requests and split status/action paths follow their documented conservative - seams, including already-hidden and no-alert outcomes; -16. failed or timed-out existing-session open/relaunch leaves the old frame expired after dispatch; -17. a read-only ref with retained stale evidence returns the structured warning, while missing frame - evidence fails without falling through to a newer observation by positional coincidence; -18. a large-tree fixture proves frame/observation separation introduces no deep copy and records the - bounded peak retained nodes/bytes; and -19. enforcement and error shapes are consistent across supported platforms. - -Performance regression tests prove that admission, expiration, and frame/observation separation add -no implicit capture or provider/platform call. Device benchmarks are required only if those tests or -live evidence reveal a regression. No path may claim success until a test proves the relevant -direct/native/runtime/provider branch actually executed. - -Before enforcement is enabled for a platform, fresh-build live evidence must exercise every supported -production seam affected there: Apple runtime-ref and direct/native paths where available, Android -helper freshness and recovery, generic/lifecycle mutation, and at least one real provider-backed -interaction plus lifecycle operation. Each run must prove a fresh ref succeeds, a second mutation with -the stale ref is rejected before dispatch, and a fresh observation restores usability. A seam not -exercised remains disabled or is recorded as an explicit release blocker. Fixture-backed tests and -registry claims are necessary but do not substitute for this live evidence. +Completion requires contract and provider tests that prove: + +1. After an unobserved first mutation, bare and pinned second mutations reject. +2. Definitive pre-seam validation, target, and replay-identity failures preserve the frame. +3. Timeout, cancellation, connection loss, and other post-seam failures leave it expired. +4. Every listed path crosses the same seam: runtime ref/selector/coordinate, native ref, direct selector, provider-backed interaction, and provider-backed lifecycle/generic paths. +5. Android freshness cannot retarget admitted refs by positional coincidence. +6. Read-only operational captures do not replace authorized frames. +7. Partial publication admits only emitted pinned refs and reports each typed failure. + It cannot authorize unrelated plain refs. + CLI text pins every reusable partial ref. + JSON and Node.js retain response-level representation. +8. Complete publication admits its plain namespace. + Sparse/unusable captures do not. +9. Mutating `find` omits `refsGeneration` and never pins or activates its pre-action ref as post-action state. + After MCP remembers a snapshot pin and performs mutating `find`, later ref input still forwards the old pin. + The daemon rejects it as stale. +10. Batch and replay delegate per step and retain failure/partial-result contracts. + They permit `snapshot -> @ref` with an intermediate digest without exposed pins or dynamic bindings. +11. Recorded scripts contain selectors instead of portable generation claims. +12. Press/swipe series and Apple chunks resolve and expire exactly once. +13. Android dialog recovery expires before its first side effect. + After recovery mutates, it aborts an outstanding ref action. +14. Consecutive `snapshot -s @ref` calls resolve the same admitted semantic scope after ref reindex. + Mutation, unrelated publication, or another scope clears lineage. + Arbitrary stale refs cannot use it. + Empty/unusable repetitions preserve the last useful frame and still-consecutive lineage. +15. Fused keyboard/alert and split status/action requests follow their conservative seams, including already-hidden and no-alert outcomes. +16. Failed or timed-out existing-session open/relaunch leaves the old frame expired after dispatch. +17. Read-only refs with retained stale evidence return structured warnings. + Missing evidence fails without positional fallback to newer observations. +18. A large-tree fixture proves that frame/observation separation adds no deep copy. + It records bounded peak retained nodes/bytes. +19. Enforcement and error shapes are consistent across supported platforms. + +Performance regression tests prove that admission, expiry, and frame/observation separation add no implicit capture or provider/platform call. +Device benchmarks are required only if tests or live evidence reveal regression. +No path may claim success until a test proves execution of the relevant direct/native/runtime/provider branch. + +Before platform enforcement, fresh-build live evidence must exercise every affected supported production seam: + +- Apple runtime-ref and direct/native paths where available. +- Android helper freshness and recovery. +- Generic/lifecycle mutation. +- At least one real provider-backed interaction and lifecycle operation. + +Every run must prove: + +1. A fresh ref succeeds. +2. A second mutation with the stale ref rejects before dispatch. +3. A fresh observation restores usability. + +Unexercised seams remain disabled or explicit release blockers. +Fixture tests and registry claims are necessary but do not replace live evidence. ## Alternatives considered -- **Stable refs across trees:** rejected because positional accessibility snapshots have no durable - cross-tree identity, and semantic IDs are neither universal nor unique. -- **Expire only after successful mutation:** rejected because a timeout or transport failure may occur - after the device acted. -- **Keep one snapshot for observation and authorization:** rejected because read-only internal captures - can reindex or invalidate caller refs. -- **Automatically capture after every mutation:** rejected because it adds the dominant platform cost, - can itself fail or be sparse, and hides rather than defines lifetime. -- **Retain every generation's full tree:** rejected as unbounded session memory and unnecessary for a - single-current-frame contract. -- **Add a per-ref historical ledger immediately:** rejected until evidence requires concurrent - generation support; one bounded current frame plus issuance scope is sufficient. -- **Derive the policy from the runner's command traits:** rejected because those traits classify a - wire command for what the runner may still do about its app, while daemon ref authorization - classifies a command for what it did to the frame. -- **Add batch interpolation or an unsafe ref-stability override:** rejected as a new orchestration - interface that bypasses the same safety rule. -- **Force lifetime into ADR 0011's element path matrix:** rejected because ref lifetime spans commands, - time, non-element invalidators, and orchestrators; the matrix remains the enforcement record for - per-path element guarantees. +- **Stable refs across trees.** Rejected because positional snapshots have no durable cross-tree identity. + Semantic IDs are neither universal nor unique. +- **Expire only after successful mutation.** Rejected because the device may act before timeout or transport failure. +- **Use one snapshot for observation and authorization.** Rejected because internal read-only captures can reindex or invalidate caller refs. +- **Automatically capture after every mutation.** Rejected because it adds the dominant platform cost and can fail or return sparse output. + It hides lifetime instead of defining it. +- **Retain every generation's full tree.** Rejected as unbounded memory unnecessary for one current frame. +- **Immediately add per-ref history.** Rejected until evidence requires concurrent generations. + One bounded current frame and issuance scope suffice. +- **Derive policy from runner traits.** Rejected because runner traits classify wire commands for remaining app work. + Daemon ref authorization classifies their effects on the frame. +- **Add batch interpolation or unsafe ref-stability overrides.** Rejected as new orchestration interfaces that bypass the safety rule. +- **Put lifetime in ADR 0011's element path matrix.** Rejected because lifetime spans commands, time, non-element invalidators, and orchestrators. + The matrix retains per-path element guarantees. From 62e37833ed4d70d499e2ab10ed44fd6042e49387 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:08:34 +0200 Subject: [PATCH 22/55] docs(adr): write ADR 0015 in ASD-STE100 --- docs/adr/0015-direct-maestro-engine.md | 422 +++++++++++++------------ 1 file changed, 226 insertions(+), 196 deletions(-) diff --git a/docs/adr/0015-direct-maestro-engine.md b/docs/adr/0015-direct-maestro-engine.md index ac124de178..bcbcebb2f6 100644 --- a/docs/adr/0015-direct-maestro-engine.md +++ b/docs/adr/0015-direct-maestro-engine.md @@ -4,220 +4,250 @@ Accepted +## Rules at a glance + +- Parse supported YAML into a source-preserving typed Maestro program. Execute it through a narrow compatibility runtime port. +- Do not route Maestro programs through `SessionAction`, private command names, positional JSON, or recursive replay dispatch. +- Keep variables and observation generation in one execution context. Every mutation attempt invalidates retained evidence before dispatch. +- Keep geometry action-local. Do not reuse it across command boundaries. +- Preserve upstream first-match/index semantics and the stated public-command ambiguity exception. +- Defer pending stabilization to the next mutation. Authored observations poll their own conditions immediately. +- Preserve the performance contract and declare conformance divergences. +- Keep `${...}` interpolation lookup-only. Gate `evalScript` through `MaestroEngineOptions.trustedScripts`. +- Do not ship two production engines or a runtime fallback between them. + ## Context -At decision time, Maestro YAML compiled into generic `SessionAction` strings. Commands whose semantics did not -match native replay become private `__maestro*` names with positional payloads, then replay dispatch -routes those strings through a second Maestro switch before recursively invoking ordinary daemon -commands. Compatibility state is split between the replay variable scope and three `WeakMap` caches. +At decision time, Maestro YAML compiled to generic `SessionAction` strings. +Commands incompatible with native replay became private `__maestro*` names with positional payloads. +Replay dispatch routed those strings through a second Maestro switch, then recursively invoked ordinary daemon commands. +Replay variable scope and three `WeakMap` caches divided compatibility state. -That path grew a second selector resolver, several polling loops, action-after-assertion recovery, and -gesture-coordinate adapters. The indirection makes successful responses hard to prove: a fast native -wait may establish text existence while the compatibility assertion requires a visible Maestro target. -It also hides authored coordinate space until runtime positional decoding. +This path acquired a second selector resolver, several polling loops, action-after-assertion recovery, and gesture-coordinate adapters. +The indirection makes successful responses hard to prove. +For example, a fast native wait can prove text existence while compatibility requires a visible Maestro target. +Positional decode also hides authored coordinate space until runtime. -The inbound implementation was about 5,000 production lines and 5,000 focused test lines. -Android compatibility is materially faster than native Maestro in the pager and navigation suites; -that advantage is a product constraint, not expendable migration headroom. +The incoming implementation had approximately 5,000 production lines and 5,000 focused test lines. +Android compatibility is materially faster than native Maestro in pager and navigation suites. +This advantage is a product constraint, not spare migration capacity. -ADR 0013 separately owns public gesture normalization, contact topology, trajectory planning, and -native injection. Maestro's supported gesture surface is single-pointer swipe only. Maestro swipes -normalize to ADR 0013's canonical `pan` input (origin, delta, durationMs) and carry the -`endpoint-hold` execution profile through a daemon-internal compatibility seam -(`internal.gestureExecutionProfile`), preserving iOS fast-swipe-then-hold behavior without exposing -the profile on the public command surface. +ADR 0013 owns public gesture normalization, contact topology, trajectory plans, and native injection. +Maestro supports only single-pointer swipe gestures. +These normalize to canonical `pan` with origin, delta, and durationMs. +Daemon-internal `internal.gestureExecutionProfile` carries `endpoint-hold`. +This preserves iOS fast-swipe-then-hold behavior without a public execution-profile option. ## Decision -Parse supported Maestro YAML into a source-preserving typed program and execute it directly through a -narrow compatibility runtime port. Do not lower the program through `SessionAction`, private command -names, positional JSON, or a recursive replay dispatcher. The daemon adapter may invoke ordinary public -commands as typed operations; those calls reuse the shared command semantics and never re-enter Maestro -parsing or compatibility dispatch. +Parse supported Maestro YAML into a source-preserving typed program. +Execute it directly through a narrow compatibility runtime port. +Do not lower it through `SessionAction`, private command names, positional JSON, or recursive replay dispatch. +The daemon adapter may invoke ordinary public commands as typed operations. +Those calls reuse shared command semantics without re-entry into Maestro parsing or compatibility dispatch. The engine has five responsibilities: -1. The parser validates the supported grammar and preserves source path and line on every command. -2. The interpreter owns hooks, includes, environment scopes, conditions, repeat/retry, and ordered - command execution. Runtime repeat/retry blocks remain compact plan nodes; the plan does not expand - authored iteration counts. `repeat.times` preserves the authored count, while retry clamps - `maxRetries` to upstream's three-retry preset (four total attempts). -3. The runtime port exposes typed app, input, observation, target, and single-pointer gesture - operations backed by the existing agent-device runtime and platform adapters. -4. One explicit execution context owns variables and the current observation generation. A mutation - invalidates that generation; reads may reuse semantic evidence only within the same generation. - Geometry is action-local and is never carried across command boundaries. -5. An observer adapts source-aware progress, traces, artifacts, and failures to the existing replay and - test result contracts. Observer telemetry is redacted and best-effort; trace persistence cannot - change command success or failure. - -Maestro `stopApp` terminates its target through a daemon-internal app-only close and preserves the -owning replay session. Public `close` remains the session finalizer that releases recordings, runner -state, leases, and stored runtime hints. - -The engine does not implement platform input. Absolute and percentage swipes preserve authored -endpoints without a hierarchy capture. Directional horizontal swipes reuse ADR 0013's shared in-page -preset geometry so an iOS right swipe does not become an interactive-back gesture; vertical presets -retain Maestro's platform geometry. All viewport-relative swipes resolve the cheapest fresh interaction -viewport available so ADR 0013 can validate every planned sample. Maestro swipes pair that resolved -viewport with `internal.gestureExecutionProfile: 'endpoint-hold'` on the nested public `gesture` request -so ADR 0013 produces the iOS fast-swipe-then-hold profile without a public `executionProfile` field. -When normalization already resolves a viewport, the adapter pairs it with the nested public gesture -request as daemon-internal metadata. ADR 0013 planning consumes that exact frame instead of probing -the platform a second time. - -Target-relative swipes reuse the target-resolution observation. The resulting typed single-pointer -motion enters ADR 0013 as the canonical `pan` input, with the `endpoint-hold` execution profile carried -as daemon-internal metadata. Maestro code cannot construct or execute two-pointer pan, pinch, rotate, -transform, or physical pointer trajectories. - -Simple successful target queries return their match, visibility decision, candidate count, and -observation generation in one response. The daemon may retain the provider snapshot behind that -evidence, but geometry stays action-local. Raw hierarchies, screenshots, and complete candidate lists -are failure/debug artifacts, not happy-path requirements. - -The daemon adapter may retain the provider snapshot behind a successful observation without exposing it -through the engine contract. A following target resolution may use that snapshot only as semantic -evidence. Maestro selector semantics use the shared structurally normalized provider snapshot, before -iOS canonical interactive presentation. They deliberately do not request the public `snapshot --raw` -shape, whose output contract bypasses shared pruning and overlay annotations. On iOS, the adapter derives the provider's canonical -interactive presentation from that same snapshot without another capture. Raw matching still selects -the node and authored index; when that same source node has canonical interactive bounds, tap-like -actions use those bounds. A unique exact, canonically hittable match may be dispatched with its -resolved point so XCTest binds the live selector identity and coordinate delivery atomically; a -structured live-selector miss, ambiguity, point mismatch, or off-screen result -falls back to fresh Maestro resolution. All other targets capture fresh geometry before coordinate -dispatch. Maestro visibility does not apply agent-device's React Native overlay blocking policy; -debug-overlay handling remains an explicit public-command workflow. Visibility can -be true while a scroll view or tab strip is still moving, so an observation frame is never authoritative -for a later interaction, even within the same mutation generation. Every mutating attempt invalidates -retained evidence before dispatch, including an attempt whose dispatch reports failure, because the -adapter cannot prove the app stayed unchanged. - -Maestro target resolution preserves upstream first-match and explicit-index semantics. Equivalent iOS -accessibility wrapper chains are normalized to the semantic control or deepest matching leaf before -applying an authored index, matching XCTest element-query behavior without changing distinct nested -matches. It does not adopt the public agent-device command surface's unique-match requirement. Conversely, an -`AMBIGUOUS_MATCH` produced by a nested public command is an agent-device authoring failure and is not -suppressed by Maestro `optional`; atomic iOS dispatch handles that result by performing fresh Maestro -resolution, as described above. Cancellation and infrastructure failures are likewise non-optional. - -Mutations that can leave an in-flight transition record a pending stabilization boundary. Before another -mutating command, the runtime samples at the compatibility polling cadence until the hierarchy is stable -and retains that final observation for the next read. Commands with specialized completion semantics settle -inline instead. Hierarchy signatures project the available semantic attributes, tree topology, and -fixed-order integer edges while excluding provider-only metadata, object-key order, and subpixel noise. - -An authored observation after a mutation skips the pending barrier and polls its own condition immediately. -Its successful snapshot becomes the baseline for the boundary but does not discharge mutation ordering. If -another mutation follows, the runtime waits one polling interval and compares a fresh hierarchy with that -baseline, continuing until stable when the UI is still changing. If no later mutation follows, it performs no -extra capture. This intentionally differs from upstream Maestro, which settles before evaluating an assertion, -and retains the previous agent-device compatibility engine's faster read behavior without allowing a later -tap, gesture, scroll, or input to overtake an in-flight transition. - -`retryTapIfNoChange` defaults to false. When explicitly enabled, the runtime compares the stable hierarchy -with the target-resolution hierarchy. On iOS, an unchanged hierarchy triggers the same screenshot comparison -used by Maestro; the runtime retries once only when both surfaces remain unchanged. Screenshot evidence is -best-effort because a failed capture cannot prove that repeating a mutation is safe. The stable hierarchy -primes the next command, so the retry policy does not add another hierarchy read there. - -`waitForAnimationToEnd` uses its own screenshot-stability operation, matching upstream's two immediate -captures per attempt and 0.005% normalized absolute RGB-difference threshold. These captures explicitly -bypass ordinary screenshot stabilization so the command observes the application rather than -recursively waiting on another settling policy. After a successful authored visual wait, screenshot -stability discharges the same-generation pending mutation boundary instead of preceding the visual -barrier with a hierarchy capture. Reaching the visual deadline without observing a stable pair leaves -that boundary pending for the next mutating command's hierarchy settle. On iOS, -comparison captures use the persistent runner's screenshot surface so both frames come from one warmed -transport and avoid simulator screenshot setup between polls. - -Upstream Maestro is a version-pinned development reference, not a production dependency. Conformance is -enforced by the three-layer oracle in `scripts/maestro-conformance` (issue #1274), which replaced the -original hand-typed parser fixture. Layer 1 drives the pinned `maestro-orchestra` YAML parser -(`dev.mobile:maestro-orchestra:2.5.1`, SHA-verified at regeneration) over a corpus of vendored upstream -flows plus targeted bug-class flows, capturing each parse; the deterministic verifier replays that -capture against our engine and classifies every flow as identical / both-reject / we-reject / mismatch, -requiring each non-identical outcome to be a declared, on-the-record divergence. Layer 2 reads upstream -constants (retry cap, animation-wait threshold/timeout, erase cap) and parser-observed defaults (swipe -duration) straight from the pinned bytecode and cross-checks each against the live `MAESTRO_COMPATIBILITY_PRESETS` -constant. Both generated layers are checked in and verified in CI via `node --test` with no Java. Because -per-PR CI cannot re-derive them, each fixture carries a `contentHash` seal that the verifier recomputes, -so a hand edit to a captured command or constant fails the build; the scheduled `conformance-regenerate` -job then re-runs the harness against the pinned jars and fails on any byte difference, which is what makes -"generated from upstream" enforced rather than documented. Layer 3 runs a small set of app-observable -differential scenarios (settle ordering, tap retries, optional warned-vs-failed) through both engines -against the real fixture app, pinning the Maestro CLI to the same version as layers 1-2; it is scheduled, -never per-PR. Layer 3 carries the same declared-divergence contract as layer 1: a scenario that currently -fails declares a `knownDivergence` with a required tracking issue, the schedule stays green on that known -gap, and only undeclared divergences fail. A declaration that stops reproducing also fails, so the fix PR -must remove it and the differential becomes the acceptance test for its own findings. This exists because -the instrument must not be blocked on repairing what it just measured. The four #1217 regressions (percent -rounding/rejection, target-swipe direction default, retry-cap semantics, settle-loop ordering) each have a -dedicated fixture. Percent truncation's runtime half is pinned by a pure unit test rather than a device -scenario, because a one-pixel delta is not app-observable. Bug class 4's detector is an engine-side timing -invariant, unit-tested against synthetic traces, that executes on the device path — currently blocked by a -declared `scrollUntilVisible` divergence (#1299) that the differential itself found on its first working run. - -Documented intentional deviations are encoded as expected-divergence entries rather than silent -mismatches: the hierarchy-vs-screenshot animation wait (constants match, mechanism differs), the -horizontal-swipe in-page presets, condition-poll-as-stabilization, ambiguity strictness under `optional`, -and the omitted iOS 3s pre-tap static gate (`IOSDriver.SCREEN_SETTLE_TIMEOUT_MS`, a layer-2 reference-only -vector). +1. The parser validates supported grammar and preserves source path and line on every command. +2. The interpreter owns hooks, includes, environment scopes, conditions, repeat/retry, and ordered execution. + Runtime repeat/retry blocks remain compact plan nodes without expansion of authored iteration counts. + `repeat.times` preserves the authored count. + Retry clamps `maxRetries` to upstream's three-retry preset, for four total attempts. +3. The runtime port exposes typed app, input, observation, target, and single-pointer gesture operations. + Existing agent-device runtimes and platform adapters execute them. +4. One explicit execution context owns variables and current observation generation. + Mutation invalidates that generation. + Reads may reuse semantic evidence only within the same generation. + Geometry stays action-local and never crosses command boundaries. +5. An observer adapts source-aware progress, traces, artifacts, and failures to existing replay/test results. + Observer telemetry is redacted and best-effort. + Trace persistence cannot change command success or failure. + +Maestro `stopApp` uses a daemon-internal app-only close and preserves the owning replay session. +Public `close` remains the session finalizer for recordings, runner state, leases, and stored runtime hints. + +The engine does not implement platform input. +Absolute and percentage swipes preserve authored endpoints without hierarchy capture. +Directional horizontal swipes reuse ADR 0013's shared in-page preset geometry. +Thus, an iOS right swipe does not become an interactive-back gesture. +Vertical presets retain Maestro's platform geometry. +Every viewport-relative swipe resolves the least expensive fresh interaction viewport available. +ADR 0013 then validates every planned sample. +The nested public `gesture` request pairs that viewport with `internal.gestureExecutionProfile: 'endpoint-hold'`. +This produces iOS fast-swipe-then-hold behavior without public `executionProfile`. +If normalization already resolved a viewport, the adapter supplies that exact frame as daemon-internal metadata. +ADR 0013 consumes it without another platform probe. + +Target-relative swipes reuse the target-resolution observation. +Their typed single-pointer motion enters ADR 0013 as canonical `pan` with internal `endpoint-hold` metadata. +Maestro code cannot construct or execute two-pointer pan, pinch, rotate, transform, or physical pointer trajectories. + +Simple successful target queries return match, visibility decision, candidate count, and observation generation in one response. +The daemon may retain the provider snapshot behind the evidence. +Geometry remains action-local. +Raw hierarchies, screenshots, and full candidate lists are failure/debug artifacts, not successful-path requirements. + +The daemon adapter may retain a successful observation's provider snapshot without exposing it through the engine contract. +A following target resolution may use it only as semantic evidence. +Maestro selectors use the shared structurally normalized provider snapshot before canonical iOS interactive presentation. +They intentionally do not request public `snapshot --raw`, which bypasses shared pruning and overlay annotations. +On iOS, the adapter derives canonical interactive presentation from the same snapshot without another capture. +Raw matching selects the node and authored index. +If the same source node has canonical interactive bounds, tap-like actions use them. + +A unique exact canonically hittable match may dispatch with its resolved point. +XCTest then binds live selector identity and coordinate delivery atomically. +A structured live-selector miss, ambiguity, point mismatch, or offscreen result returns to fresh Maestro resolution. +All other targets capture fresh geometry before coordinate dispatch. +Maestro visibility does not apply agent-device's React Native overlay blocking policy. +Debug-overlay handling remains an explicit public-command workflow. +A scroll view or tab strip can remain in motion while visibility is true. +Thus, an observation frame never authorizes a later interaction, even within one mutation generation. +Before dispatch, every mutation attempt invalidates retained evidence. +This includes failed dispatch because the adapter cannot prove that the app remained unchanged. + +Maestro resolution preserves upstream first-match and explicit-index semantics. +Before authored index selection, equivalent iOS accessibility wrapper chains normalize to the semantic control or deepest matching leaf. +This matches XCTest element queries without changing distinct nested matches. +It does not adopt public agent-device unique-match requirements. +Conversely, nested public-command `AMBIGUOUS_MATCH` is an agent-device authoring failure. +Maestro `optional` does not suppress it. +Atomic iOS dispatch handles it through fresh Maestro resolution, as above. +Cancellation and infrastructure failures are also non-optional. + +Mutations that can leave transitions in flight record a pending stabilization boundary. +Before another mutation, the runtime samples at compatibility polling cadence until the hierarchy stabilizes. +It retains the final observation for the next read. +Commands with specialized completion semantics settle inline instead. +Hierarchy signatures project available semantic attributes, topology, and fixed-order integer edges. +They exclude provider-only metadata, object-key order, and subpixel noise. + +An authored observation after mutation skips the pending barrier and immediately polls its own condition. +Its successful snapshot becomes the boundary's baseline without discharging mutation order. +If another mutation follows, the runtime waits one polling interval and compares a fresh hierarchy with that baseline. +If the UI still changes, it continues until stable. +If no later mutation follows, it performs no extra capture. +Upstream Maestro instead settles before assertion evaluation. +This intentional difference retains the previous engine's faster reads while preventing later tap, gesture, scroll, or input from overtaking transitions. + +`retryTapIfNoChange` defaults to false. +If explicitly enabled, the runtime compares stable hierarchy with target-resolution hierarchy. +On iOS, unchanged hierarchy triggers Maestro's screenshot comparison. +It retries once only if both surfaces remain unchanged. +Screenshot evidence is best-effort because failed capture cannot prove safe mutation repetition. +The stable hierarchy prepares the next command without another hierarchy read. + +`waitForAnimationToEnd` owns a screenshot-stability operation. +It matches upstream's two immediate captures per attempt and 0.005% normalized absolute RGB-difference threshold. +These captures bypass ordinary screenshot stabilization to avoid a recursive wait on another stabilization policy. +After successful authored visual wait, screenshot stability discharges the same-generation pending mutation boundary. +No hierarchy capture precedes that visual barrier. +If no stable pair appears before the visual deadline, the boundary remains pending for the next mutation's hierarchy stabilization. +On iOS, comparison captures use the persistent runner's screenshot surface. +Both frames therefore use one initialized transport without simulator screenshot setup between polls. + +Upstream Maestro is a version-pinned development reference, not a production dependency. +`scripts/maestro-conformance` enforces a three-layer oracle (issue #1274) instead of a manually typed parser fixture: + +1. Layer 1 runs pinned `maestro-orchestra` YAML parser `dev.mobile:maestro-orchestra:2.5.1` over vendored upstream flows and targeted failure-class flows. + Regeneration verifies its SHA. + The harness captures each parse. + A deterministic verifier replays captures against this engine. + Every flow becomes identical / both-reject / we-reject / mismatch. + Every non-identical outcome requires an explicit declared divergence. +2. Layer 2 reads upstream retry cap, animation-wait threshold/timeout, and erase cap directly from pinned bytecode. + It also reads parser-observed swipe-duration defaults. + It compares each value with live `MAESTRO_COMPATIBILITY_PRESETS`. +3. Layer 3 runs app-observable differential scenarios through both engines on the real fixture app. + These include stabilization order, tap retries, and optional warned-vs-failed outcomes. + It pins Maestro CLI to the same version as layers 1–2. + It runs on a schedule, never per PR. + +Both generated layers are committed and verified in CI through `node --test` without Java. +Per-PR CI cannot regenerate them. +Each fixture therefore carries a `contentHash` seal recomputed by the verifier. +Manual changes to captured commands or constants fail the build. +Scheduled `conformance-regenerate` reruns the harness against pinned jars and fails on any byte difference. +Thus, checks enforce upstream generation. + +Layer 3 uses the same declared-divergence contract as Layer 1. +A failing scenario declares `knownDivergence` with a required tracking issue. +Known gaps keep the schedule green; only undeclared divergences fail. +A declaration that no longer reproduces also fails. +Thus, a fix PR must remove it, and the differential becomes its acceptance test. +The measurement instrument must remain usable before its detected defects are repaired. + +Each of the four #1217 regressions has a dedicated fixture: + +- Percent rounding/rejection. +- Target-swipe direction default. +- Retry-cap semantics. +- Stabilization-loop order. + +A pure unit test fixes percent truncation's runtime behavior because a one-pixel delta is not app-observable. +Failure class 4 uses an engine-side timing invariant on the device path, unit-tested with synthetic traces. +A declared `scrollUntilVisible` divergence (#1299) currently blocks it. +The differential itself detected that gap on its first working run. + +Expected-divergence entries encode intentional deviations: + +- Hierarchy-vs-screenshot animation wait, with matching constants but different mechanisms. +- Horizontal-swipe in-page presets. +- Condition polls used for stabilization. +- Ambiguity strictness under `optional`. +- Omitted iOS 3s pre-tap static gate, `IOSDriver.SCREEN_SETTLE_TIMEOUT_MS`, retained as a layer-2 reference-only vector. ## Performance Contract -The direct engine preserves these properties on Android and iOS across the pager and react-navigation -corpora: - -- total wall time is no slower than the pre-migration compatibility engine; -- an observation immediately after a mutation polls its authored condition directly; a later mutation uses - that result as its stabilization baseline; -- no command captures a second hierarchy merely to re-verify evidence produced within that command; -- absolute coordinate swipes perform one direct viewport query and no accessibility capture; -- mutation-to-mutation stabilization is deferred to the next command boundary and primes the following read; -- percentage swipe conversion preserves authored endpoints exactly; -- helper/runner startup remains amortized across a suite; -- p50/p95 command latency, hierarchy and screenshot captures, tap retries, and transferred hierarchy - bytes are reported separately; -- failure-only diagnostics are excluded from happy-path latency comparisons. - -Android verification must prove the bundled helper backend and version. iOS verification must separate -runner startup from warm command latency. Both platforms rerun the non-Maestro gesture canaries; their -two-pointer plans, executor selection, and app-observable effects must remain unchanged. +Across Android/iOS pager and react-navigation corpora, the direct engine preserves: + +- Total wall time no slower than the pre-migration compatibility engine. +- Immediate authored condition polls after mutation, with that result as a later mutation's stabilization baseline. +- No second hierarchy capture solely to verify evidence produced within the same command. +- One direct viewport query and no accessibility capture for absolute coordinate swipes. +- Deferred mutation-to-mutation stabilization at the next command boundary, with its result reused by the following read. +- Exact authored endpoints after percentage swipe conversion. +- Helper/runner startup amortized across a suite. +- Separate reports for p50/p95 command latency, hierarchy/screenshot captures, tap retries, and transferred hierarchy bytes. +- Failure-only diagnostics excluded from successful-path latency comparisons. + +Android verification must prove the bundled helper backend and version. +iOS verification must separate runner startup from warm command latency. +Both platforms rerun non-Maestro gesture canaries. +Their two-pointer plans, executor selection, and app-observable effects must remain unchanged. ## Consequences -- agent-device supports selected Maestro Flow syntax and behavior. Unsupported features return - explicit errors, and intentional differences are declared in the conformance fixtures. -- `${...}` interpolation stays variable-lookup-only, by decision (#1292): upstream's GraalJS - expression evaluation is not reimplemented outside `evalScript`, so unresolved or - expression-shaped payloads fail loud with source context; the escape hatches are `evalScript` - (inline expression → `output.*` leaves) and `runScript` (file compute) → `${output.x}` (consume). - `evalScript` is the one command whose payload is evaluated as JavaScript: flow env and prior - `output` leaves are bound as strings in a `node:vm` context and the assigned `output` object - folds back into the flat string-key model, so `${output.uppercaseName}` and - `${output.list.length}` resolve. `assertTrue` (#1295) is scoped to that same lookup-only subset — - literal values and bare `${VAR}` lookups evaluated with a pinned string-truthiness table. - `node:vm` is explicitly not a security sandbox — a bound context's prototype chain still - resolves to host `Object`/`Function`, so an untrusted expression can escape it. `runScript` - already carried this trust assumption for daemon-local flows; `evalScript` inherits the same - assumption but is reachable from a flow accepted over the daemon's remote HTTP surface, so - `MaestroEngineOptions.trustedScripts` gates it — set from `publicNetworkOnly` at the HTTP - boundary (`restrictRemoteHttpRequest`) — and `evalScript` is refused outright when false. -- Shipping two production engines or a runtime fallback between them is rejected because it doubles - semantic and performance ownership. -- Source provenance and runtime values stay typed through execution. -- Compatibility policy remains local while device behavior stays in shared runtimes and backends. +- agent-device supports selected Maestro Flow syntax and behavior. + Unsupported features return explicit errors. + Conformance fixtures declare intentional differences. +- By decision #1292, `${...}` interpolation remains variable-lookup-only. + Upstream GraalJS expression evaluation is not duplicated outside `evalScript`. + Unresolved or expression-shaped payloads fail explicitly with source context. + Available computation paths are: + - `evalScript`: inline expression → `output.*` leaves. + - `runScript`: file computation → `${output.x}` consumption. + Only `evalScript` evaluates its payload as JavaScript. + Flow env and prior `output` leaves bind as strings in a `node:vm` context. + Assigned `output` becomes the flat string-key model. + Thus, `${output.uppercaseName}` and `${output.list.length}` resolve. + `assertTrue` (#1295) uses the same lookup-only subset: literal values and bare `${VAR}` lookups with a fixed string-truthiness table. + `node:vm` is explicitly not a security sandbox. + A bound context's prototype chain still reaches host `Object`/`Function`, so untrusted expressions can escape it. + Daemon-local `runScript` already has this trust assumption. + `evalScript` inherits it but is also reachable through remote HTTP flows. + `MaestroEngineOptions.trustedScripts` therefore gates it. + HTTP boundary `restrictRemoteHttpRequest` sets this option from `publicNetworkOnly`. + If false, `evalScript` is refused outright. +- Two production engines or runtime fallback between them are rejected because they duplicate semantic and performance ownership. +- Source provenance and runtime values remain typed through execution. +- Compatibility policy stays local; shared runtimes/backends retain device behavior. - Cross-platform correctness may require richer provider query evidence, but not additional round trips. -- ADR 0013 can be rewritten internally without changing Maestro as long as its normalized - single-pointer boundary and executor guarantees remain available. +- ADR 0013 internals can change while its normalized single-pointer boundary and executor guarantees remain available to Maestro. ## Alternatives Considered -- Embed upstream Orchestra: rejected because Java startup, package weight, driver ownership, and - platform coverage would erase performance and backend advantages. -- Build a shared replay VM first: rejected until native `.ad` needs structured runtime control flow; - one caller does not justify a broader abstraction. -- Keep compiling to typed `SessionAction` variants: rejected because it retains the replay trampoline - and prevents the compatibility engine from owning observation generations directly. +- **Embed upstream Orchestra.** Rejected because Java startup, package weight, driver ownership, and platform coverage would remove performance and backend advantages. +- **Build a shared replay VM first.** Rejected until native `.ad` needs structured runtime control flow. + One caller does not justify a broader abstraction. +- **Continue compilation to typed `SessionAction` variants.** Rejected because it retains recursive replay dispatch and prevents direct ownership of observation generations. From 887ad689f6148e1d0bcd70ffbb2ba7e235287961 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:09:35 +0200 Subject: [PATCH 23/55] docs(adr): remove completed ADR 0016 process history --- .../0016-active-session-script-publication.md | 29 +++++++------------ 1 file changed, 11 insertions(+), 18 deletions(-) diff --git a/docs/adr/0016-active-session-script-publication.md b/docs/adr/0016-active-session-script-publication.md index 75d30af7fc..c9d8daa873 100644 --- a/docs/adr/0016-active-session-script-publication.md +++ b/docs/adr/0016-active-session-script-publication.md @@ -2,9 +2,12 @@ ## Status -Accepted; implemented on `main` (`packages/daemon/src/sessions/handlers/session-script-publication.ts`, -`packages/daemon/src/core/session-script-active-publication.ts`), including the #1349 identity-level -destination-guard amendment. +Accepted. + +Completed process history is in git history. + +Implementation owners: `packages/daemon/src/sessions/handlers/session-script-publication.ts` and +`packages/daemon/src/core/session-script-active-publication.ts`. ## Rules at a glance @@ -25,7 +28,7 @@ Normative summary; the binding contracts and refusal cases are in [Decision](#de - `ReplayCommandResult.sessionActive` (from the daemon's session store, never script re-parsing) is what lets a one-shot client keep the daemon alive on a close-less handoff. - Sensitive `fill` inputs must be recorded as placeholders via `fill --record-as ` - (ADR 0017, shipped for #1348); unparameterized `fill`/`type` values persist literally into the + (ADR 0017, #1348); unparameterized `fill`/`type` values persist literally into the artifact, so a secret entered without `--record-as` is published. The protection is recording-session-scoped (ADR 0017's #1398 amendment): a later action's own recorded evidence can never re-serialize an app-rendered echo of an already-parameterized value either. @@ -98,7 +101,7 @@ work, so the caller can retry with plain `close`. Plain `close` tears down ABORT writing; closing an unpublished ARMED recording retains the existing close-time publication behavior. A fresh session is the only re-arming boundary. -> **Amendment (2026-08-02, shipped).** A session that was never armed — no recorded +> **Amendment (2026-08-02).** A session that was never armed — no recorded > `open --save-script` at all — has no fourth pre-ARMED state name here, but it previously fell > through the same close-time write path as an ARMED recording: `close --save-script` on it folded > the request into the authoring lifecycle at record time and published anyway. Live evidence @@ -113,7 +116,7 @@ fresh session is the only re-arming boundary. > *bare* `close` (no `--save-script` on the close itself) publish; that case is resolved by the > amendments below. -> **Amendment (#1533, shipped).** ABORTED terminality above was enforced only by +> **Amendment (#1533).** ABORTED terminality above was enforced only by > `abortAuthoringOnSecondOpen` clearing `session.recordSession` — inert by ordering, not by > construction. `recordSession` is an evidence-capture flag that several surfaces set directly, so > a later `--save-script` re-armed it while the status stayed ABORTED, and a *bare* `close` then @@ -122,17 +125,7 @@ fresh session is the only re-arming boundary. > caller that plain `close` "tears down without writing", so the gap also made an existing error > message untrue. > -> ABORTED is now terminal by construction. `--save-script` arms recording through one rule owned by -> the publication projection, which answers "not recording" for an ABORTED lifecycle on every -> surface that handles the flag — the re-open builder, the close finalizer, and the recorded-action -> ingress — so the flag can no longer contradict the status. Publication authorization is likewise -> the aggregate's to answer: the writer asks one publication-blocked question covering -> not-recording, an uncommittable or committed repair, and an ABORTED authoring lifecycle alike, so -> every path that reaches it (bare `close`, teardown, idle-reap, active publication) refuses. ARMED -> and PUBLISHED lifecycles and every repair transaction are unchanged; no state name, transition, -> or entry point in the lifecycle above is added or altered. - -> **Amendment (#1533 follow-up, shipped).** The rule above still relied on a stored +> **Amendment (#1533 follow-up).** The rule above still relied on a stored > `SessionState.recordSession` that every writer had to set in step with the lifecycle. Routing the > writes through one place made the two agree; it did not make disagreement unrepresentable, and > the field remained a second source of truth for a question the aggregate already answered. @@ -177,7 +170,7 @@ the serialized guard before filesystem work and never relies on repair-only bare `session save-script` refuses publication without this **destination guard** and tells the author to record one. V1 does not infer a screen identity from a snapshot or synthesize an implicit guard. -> **Amendment (#1349, shipped).** The guard is now identity-level, not merely selector-level. A +> **Amendment (#1349).** The guard is now identity-level, not merely selector-level. A > selector wait recorded while ARMED captures landmark-mode `target-v1` evidence for the matched > element (ADR 0012's read-only-step amendment), and a qualifying destination guard is exactly a > selector wait whose annotation is `verification: "verified"` — a selector wait on an identity-empty From 455373366f33ebc8c8794cacbd32cbdbf0db749b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:10:12 +0200 Subject: [PATCH 24/55] docs(adr): write ADR 0016 in ASD-STE100 --- .../0016-active-session-script-publication.md | 571 +++++++++--------- 1 file changed, 277 insertions(+), 294 deletions(-) diff --git a/docs/adr/0016-active-session-script-publication.md b/docs/adr/0016-active-session-script-publication.md index c9d8daa873..41d7853a51 100644 --- a/docs/adr/0016-active-session-script-publication.md +++ b/docs/adr/0016-active-session-script-publication.md @@ -6,61 +6,55 @@ Accepted. Completed process history is in git history. -Implementation owners: `packages/daemon/src/sessions/handlers/session-script-publication.ts` and -`packages/daemon/src/core/session-script-active-publication.ts`. +Implementation owners are `packages/daemon/src/sessions/handlers/session-script-publication.ts` and `packages/daemon/src/core/session-script-active-publication.ts`. ## Rules at a glance -Normative summary; the binding contracts and refusal cases are in [Decision](#decision). - -- `session save-script [path] [--force]` publishes the current ordinary script recording without - closing the app or the session; the session stays live at the destination. -- Recording lifecycle: **ARMED** (only by a new session's initial `open --save-script`) -> - **ABORTED** (a second successful plain `open`) or **PUBLISHED** (successful save). ABORTED and - PUBLISHED are terminal until the session is destroyed; a filesystem/target failure leaves the - recording ARMED and retryable. -- Publication requires exactly one initial recorded `open`, a portable selector-`wait` destination - guard after the last descriptor-classified mutating action — since #1349 the guard's `target-v1` - annotation must be `verification: "verified"` — and no unresolved session-local `@ref`. Every - refusal happens before filesystem work and names its recovery. -- Repair transactions (ADR 0012) are disjoint: a session with `saveScriptBoundary` set refuses this - action without changing repair state. -- `ReplayCommandResult.sessionActive` (from the daemon's session store, never script re-parsing) is - what lets a one-shot client keep the daemon alive on a close-less handoff. -- Sensitive `fill` inputs must be recorded as placeholders via `fill --record-as ` - (ADR 0017, #1348); unparameterized `fill`/`type` values persist literally into the - artifact, so a secret entered without `--record-as` is published. The protection is - recording-session-scoped (ADR 0017's #1398 amendment): a later action's own recorded evidence can - never re-serialize an app-rendered echo of an already-parameterized value either. +[Decision](#decision) contains binding contracts and refusal cases. + +- `session save-script [path] [--force]` publishes ordinary script recording without closing the app or session. + The session remains live at the destination. +- Only a new session's initial `open --save-script` establishes **ARMED**. + A second successful plain `open` causes **ABORTED**. + Successful save causes **PUBLISHED**. + ABORTED and PUBLISHED remain terminal until session destruction. + Filesystem/target failure leaves ARMED retryable. +- Publication requires exactly one initial recorded `open`, no unresolved session-local `@ref`, and a portable selector-`wait` destination guard after the last descriptor-classified mutation. + The guard's `target-v1` annotation must have `verification: "verified"` (#1349). + Every refusal precedes filesystem work and names recovery. +- A session with `saveScriptBoundary` refuses this action without repair-state changes (ADR 0012). +- The daemon session store supplies `ReplayCommandResult.sessionActive`. + Clients use this signal to retain the daemon after a close-less handoff. +- Sensitive `fill` values must use `fill --record-as ` (ADR 0017, #1348). + Unparameterized `fill`/`type` values persist literally, so an entered secret without `--record-as` is published. + Protection applies across the recording session (#1398). + Later actions cannot serialize app-rendered echoes of already-parameterized values. ## Context -The replay repair study found re-recording a drifted journey cheaper than repairing it in two of the -three measured drift classes. The immediate reusable unit is an **open-to-destination script**: a -self-contained `.ad` script that opens the app, performs the complete journey to screen X, verifies a -destination landmark, and leaves the app session active there so an agent can continue with new work. - -Ordinary script recording currently combines two concerns. `open --save-script[=]` arms -recording before the first interaction so selector chains and ADR 0012 `target-v1` identity evidence -are captured at action time. `close` publishes the accumulated script, records a terminal `close`, and -tears down the session. That artifact is suitable as a closed test flow, but not as a live starting -state. - -Publishing only at the end of an ordinary unarmed session is insufficient. Session history retains the -commands, but the resolved target tree needed for `target-v1` evidence is deliberately discarded after -each action. Late publication can serialize selector text; it cannot reconstruct which element was -actually acted on. Making identity capture unconditional for every session would change normal -interaction execution: recording disables direct selector fast paths when a capture-backed route is -required for evidence. That performance and data-retention change has not been measured. - -Issue [#1346](https://github.com/callstack/agent-device/issues/1346) and the throwaway -[`session-save-script` prototype](https://github.com/callstack/agent-device/tree/agent/prototype-session-save-replay/scripts/prototypes/session-save-script) -record the motivating workflow and API trial. Composable lifecycle-free fragments remain a separate -decision in [#1336](https://github.com/callstack/agent-device/issues/1336). +The replay repair study found complete re-records cheaper than repairs in two of three measured drift classes. +The immediate reusable unit is an **open-to-destination script**. +This self-contained `.ad` script opens an app, reaches screen X, verifies a destination landmark, and leaves the session active for further work. + +Ordinary recording combines two concerns. +`open --save-script[=]` arms recording before the first interaction. +It captures selector chains and ADR 0012 `target-v1` identity evidence at action time. +`close` publishes accumulated actions, adds terminal `close`, and tears down the session. +That artifact serves closed test flows, not live starting states. + +Late publication from an ordinary unarmed session is insufficient. +History retains commands but intentionally discards the resolved target tree after each action. +Late publication can serialize selector text but cannot reconstruct the actual target identity. +Unconditional identity capture would change normal execution. +Recording disables direct selector fast paths when evidence requires a capture-backed route. +This performance and data-retention change remains unmeasured. + +Issue [#1346](https://github.com/callstack/agent-device/issues/1346) and the disposable [`session-save-script` prototype](https://github.com/callstack/agent-device/tree/agent/prototype-session-save-replay/scripts/prototypes/session-save-script) record the workflow and API trial. +Composable lifecycle-free fragments remain a separate decision in [#1336](https://github.com/callstack/agent-device/issues/1336). ## Decision -Add an explicit publication action for an already-armed ordinary script recording: +Add explicit publication for an already-armed ordinary recording: ```sh agent-device open com.example.app --relaunch --save-script=screen-x.ad @@ -69,280 +63,269 @@ agent-device wait 'role="heading" label="Screen X"' agent-device session save-script ``` -`session save-script [path] [--force]` publishes the current ordinary script recording without -closing the app or deleting the session. An explicit `path` retargets the recording using the existing -target/force authorization rules. Without it, publication uses the path armed by `open --save-script` -or the existing generated default. +`session save-script [path] [--force]` publishes without app close or session deletion. +An explicit `path` retargets recording under existing target/force authorization rules. +Without it, publication uses the path from `open --save-script` or the existing generated default. ### Recording lifecycle -An ordinary recording eligible for active-session publication has three states: - -- **ARMED**: established only when a new session's initial successful - `open --save-script[=]` is recorded as action zero. The session records portable action inputs - and fresh target identity evidence. -- **ABORTED**: reached when another plain `open` succeeds while ARMED. The app operation may continue, - but the recording is no longer a single-open bootstrap and close-time publication is disarmed. The - successful `open` response warns that a fresh session is required to author another script. -- **PUBLISHED**: reached only after `session save-script` atomically commits the complete history from - the sole recorded `open` through the current action. The session remains active at the destination, - but close-time script publication is disarmed. - -A filesystem or target-collision failure leaves the recording ARMED, including its path and same-target -`--force` authorization, so the caller can correct the target or permissions and retry. ABORTED and -PUBLISHED are terminal until the session is destroyed. `session save-script` in either state fails -loudly and never writes. - -Every existing arming entry point respects terminality. `open --save-script` on any existing session — -unarmed, ARMED, ABORTED, or PUBLISHED — is rejected before app dispatch; a plain later `open` is allowed, -causing ARMED to become ABORTED while leaving ABORTED/PUBLISHED unchanged. -`close --save-script[=]` in ABORTED or PUBLISHED is rejected before platform close or filesystem -work, so the caller can retry with plain `close`. Plain `close` tears down ABORTED/PUBLISHED without -writing; closing an unpublished ARMED recording retains the existing close-time publication behavior. A -fresh session is the only re-arming boundary. - -> **Amendment (2026-08-02).** A session that was never armed — no recorded -> `open --save-script` at all — has no fourth pre-ARMED state name here, but it previously fell -> through the same close-time write path as an ARMED recording: `close --save-script` on it folded -> the request into the authoring lifecycle at record time and published anyway. Live evidence -> showed this produces a script whose actions carry selector fallback chains but no recording-time -> `target-v1` evidence, with no signal to the caller that evidence capture never ran — degraded -> replay verification is worse than a loud refusal. `close --save-script` on a never-armed session -> is now rejected before any teardown or filesystem work, the same way ABORTED/PUBLISHED are, -> naming `open --save-script` as the recovery; a plain `close` still tears the session down -> without writing. This is distinct from -> [#1533](https://github.com/callstack/agent-device/issues/1533), which is about an -> already-ARMED-then-ABORTED session whose flag ingress re-enabled recording and let a -> *bare* `close` (no `--save-script` on the close itself) publish; that case is resolved by the -> amendments below. - -> **Amendment (#1533).** ABORTED terminality above was enforced only by -> `abortAuthoringOnSecondOpen` clearing `session.recordSession` — inert by ordering, not by -> construction. `recordSession` is an evidence-capture flag that several surfaces set directly, so -> a later `--save-script` re-armed it while the status stayed ABORTED, and a *bare* `close` then -> published the full session log through a writer that only knew how to refuse repair -> transactions. The refusal this ADR specifies for `close --save-script` in ABORTED promises the -> caller that plain `close` "tears down without writing", so the gap also made an existing error -> message untrue. -> -> **Amendment (#1533 follow-up).** The rule above still relied on a stored -> `SessionState.recordSession` that every writer had to set in step with the lifecycle. Routing the -> writes through one place made the two agree; it did not make disagreement unrepresentable, and -> the field remained a second source of truth for a question the aggregate already answered. -> -> Recording is now **derived**, not stored. `isRecordingPublication` reads it off the lifecycle — -> ordinary authoring records only while ARMED; a repair transaction records for its whole lifetime, -> terminal statuses included — and the field is gone. Three consequences worth stating, because -> they are what the derivation buys: -> -> - No surface can arm recording without moving the lifecycle that authorizes it, so #1533's drift -> is unrepresentable rather than guarded against. `buildNextOpenSession` and the close finalizer -> make no recording decision at all now. -> - The writer's publication gate is answered entirely by the aggregate. Its separate ABORTED check -> is gone, because a terminal authoring lifecycle is already not recording — no second gate that -> could disagree with the first about *authoring*. -> - Evidence capture and publication authorization now derive from the same aggregate, but they -> remain **distinct predicates**, and deliberately so. They coincide for ordinary authoring: -> ARMED both records and publishes, ABORTED and PUBLISHED do neither. They do NOT coincide for -> repair — `isRecordingPublication` is true for every repair status, `committed` and `aborted` -> included, while publication additionally requires `isRepairArmedWriteBlocked` to pass, which -> refuses a committed transaction (idempotent no-op) and one that is not yet committable (ADR -> 0012 decision 6, C2). A repair that captures evidence is therefore not necessarily a repair -> that may publish, and collapsing the two would silently republish or commit a prefix. -> -> This is behavior-preserving: the derivation reproduces exactly what the flag held at every -> transition. Whether a *committed* repair should still capture recording-time evidence is a real -> question this deliberately does not answer — the old flag said yes, and changing that is a -> behavior change, not a derivation. - -This lifecycle is distinct from ADR 0012's repair transaction. `session save-script` rejects a session -with `saveScriptBoundary` set and directs the caller to finish or abort the repair through its existing -`replay --from` and teardown commit protocol. Active-session publication never marks a repair COMPLETE, -commits a healed slice, writes `# agent-device:heal-complete`, or changes repair tombstone semantics. +An eligible ordinary recording has three states: + +- **ARMED**: only a new session's initial successful `open --save-script[=]`, recorded as action zero, establishes this state. + The session records portable inputs and fresh target identity evidence. +- **ABORTED**: another successful plain `open` while ARMED establishes this state. + The app operation may continue. + The recording no longer represents a single-open bootstrap, and close-time publication is disarmed. + The successful `open` warns that another script requires a fresh session. +- **PUBLISHED**: successful atomic `session save-script` commits complete history from the sole recorded `open` through the current action. + The session remains active at the destination, with close-time publication disarmed. + +Filesystem or target-collision failure preserves ARMED, path, and same-target `--force` authorization. +The caller can correct target or permissions and retry. +ABORTED and PUBLISHED remain terminal until session destruction. +In either state, `session save-script` fails explicitly without a write. + +Every arming entry point respects terminality. +On any existing session, `open --save-script` rejects before app dispatch. +This includes unarmed, ARMED, ABORTED, and PUBLISHED sessions. +A later plain `open` is permitted. +It changes ARMED to ABORTED and preserves ABORTED/PUBLISHED. +In ABORTED or PUBLISHED, `close --save-script[=]` rejects before platform close or filesystem work. +The caller can retry plain `close`. +Plain `close` tears down those states without a write. +Close of unpublished ARMED recording retains existing close-time publication. +Only a fresh session permits re-arming. + +**Amendment (2026-08-02).** A never-armed session has no fourth pre-ARMED state name here. +Late close-time arming can produce selector fallback chains without recording-time `target-v1` evidence. +Live evidence showed that callers received no signal that identity capture never ran. +Degraded replay verification is worse than explicit refusal. +Thus, `close --save-script` on never-armed sessions rejects before teardown or filesystem work. +It names `open --save-script` as recovery. +Plain `close` still tears down without a write. +This differs from [#1533](https://github.com/callstack/agent-device/issues/1533), which concerns an ARMED-then-ABORTED session. + +**Amendment (#1533).** A mutable `recordSession` evidence flag could disagree with ABORTED lifecycle state. +A later `--save-script` could enable evidence capture while status remained ABORTED. +A bare `close` could then publish through a writer that refused only repair transactions. +This contradicted the promised plain-close behavior after `close --save-script` refusal. +Clearing the flag in `abortAuthoringOnSecondOpen` depended on operation order instead of state construction. + +**Amendment (#1533 follow-up).** Recording is **derived**, not stored. +Routing flag writes through one owner still allowed a second source of truth to disagree with lifecycle state. +The stored `SessionState.recordSession` field is removed. +`isRecordingPublication` derives evidence capture from lifecycle: + +- Ordinary authoring records only while ARMED. +- Repair transactions record throughout their lifetime, including terminal statuses. + +Consequences of this derivation are: + +- No surface can arm recording without a lifecycle transition that authorizes it. + `buildNextOpenSession` and the close finalizer make no recording decision. +- The aggregate answers the writer's publication gate. + Every write path uses that gate, including bare `close`, teardown, idle-reap, and active publication. + A separate ABORTED check is unnecessary because terminal authoring already means no recording. +- Evidence capture and publication authorization share the aggregate but remain **distinct predicates**. + For ordinary authoring, ARMED records and publishes; ABORTED/PUBLISHED do neither. + For repair, `isRecordingPublication` remains true even for `committed` and `aborted`. + Publication additionally requires `isRepairArmedWriteBlocked` to permit the write. + It refuses committed transactions as idempotent no-ops and transactions not yet committable (ADR 0012 decision 6, C2). + +A repair that captures evidence cannot necessarily publish. +Combining these predicates could republish committed transactions or commit an incomplete prefix. +The derivation preserves the former flag's behavior at every transition. +Whether committed repairs should still capture recording-time evidence remains unanswered. +The former flag permitted it. +Changing it would change behavior, not merely derive state. + +This lifecycle is distinct from ADR 0012 repair transactions. +If `saveScriptBoundary` is set, `session save-script` rejects. +Recovery uses the existing `replay --from` and teardown commit protocol to finish or abort repair. +Active publication never: + +- Marks repair COMPLETE. +- Commits a healed slice. +- Writes `# agent-device:heal-complete`. +- Changes repair tombstone semantics. ### Destination readiness and replay handoff -The destination is an authored postcondition, not the last navigation action. Before publication, the -recorded suffix after the last mutating action must contain a `wait` whose target is a portable selector -or selector chain identifying a landmark on the ready destination screen. A duration wait, `wait stable`, -or `wait @ref` does not qualify, though `wait stable` may follow the landmark wait. The publisher validates -the serialized guard before filesystem work and never relies on repair-only bare-ref rejection. -`session save-script` refuses publication without this **destination guard** and tells the author to -record one. V1 does not infer a screen identity from a snapshot or synthesize an implicit guard. - -> **Amendment (#1349).** The guard is now identity-level, not merely selector-level. A -> selector wait recorded while ARMED captures landmark-mode `target-v1` evidence for the matched -> element (ADR 0012's read-only-step amendment), and a qualifying destination guard is exactly a -> selector wait whose annotation is `verification: "verified"` — a selector wait on an identity-empty -> element (no id and no label) records no annotation and does not qualify, so publication refuses it -> with a recovery hint naming a labeled/id-bearing landmark. On replay, polling is preserved: the wait -> keeps polling until a selector match carries the recorded identity (local identity + leaf-anchored -> ancestry prefix), and a deadline reached with only same-selector impostors fails closed as an -> `identity-mismatch` `REPLAY_DIVERGENCE` before the wait reports success. The reshuffled-screen -> false-pass below is covered by a provider-scenario regression (record → publish → replay against a -> reshuffled tree whose same-label node sits under a different id/ancestry). -> -> **Amendment (#1398, ADR 0017).** An identity-empty landmark is not the only case that fails to -> qualify: a selector wait whose only identity is an app-rendered echo of a literal the recording -> session already parameterized via `fill --record-as` also records no annotation (ADR 0017's -> session-scoped echo protection), so it does not qualify as a destination guard either. Publication -> refuses it with the same recovery hint, directing the author to a stable, non-value-bearing landmark -> — an enforced version of exactly the fix issue #1398's motivating scenario applied by hand. - -The original selector-level caveat, retained as context: the guard proved that an element matching its -selector exists, not that it is the same landmark element observed while authoring, so a reshuffled -screen containing the same weak label elsewhere could false-pass. -[#1349](https://github.com/callstack/agent-device/issues/1349) owned the identity design for waits and -remaining read-only steps. It preserves polling: ADR 0012's pre-action `target-v1` -verification cannot be attached to a wait unchanged because a not-yet-present landmark is the expected -starting condition, so the identity check runs after the wait's selector resolves and before the wait -reports success (see ADR 0012's #1349 amendment for the mechanism and the read-only-step coverage -classification). - -The phrase "last mutating action" is derived from a request-sensitive recording-effect trait on the -central `CommandDescriptor`, required for every command that records session actions and guarded by a -completeness test. It is not a publisher-local command-name set. The trait distinguishes app-state -mutation from observation for subcommands such as read-only versus mutating `find`, `keyboard`, and -`alert` actions; the existing conservative `refFrameEffect: may-invalidate` classification is not precise -enough for this boundary. - -On consumption, a script without `close` preserves the existing replay behavior: the named session stays -active and the successful `ReplayCommandResult` returns its `session` id. The caller binds subsequent -commands to that returned id. Replay reports success only after the destination guard completes; the -absence of `close` changes neither action dispatch nor when replay reports success. - -**Response shape (issue #1384).** `ReplayCommandResult` carries a required `sessionActive: boolean`, -computed from whether `session` still exists in the daemon's own store when the response is built — never -by re-parsing the script for `close`. This is what lets the real CLI/IPC client (not just the in-process -handler) actually honor "the session stays active": without an explicit signal in the response, the -client's one-shot `replay`/`test` teardown had no way to distinguish a still-active handoff from a -finished one, and tore the owning daemon down regardless (`src/daemon-client/daemon-client-lifecycle.ts`). -`sessionActive` is `true` for every close-less run (including a `--from` resume) and `false` once the -script's terminal `close` — or a repair-armed run's deferred equivalent — has executed. +The destination is an authored postcondition, not the last navigation action. +After the last mutating action, the recorded suffix must contain a portable selector or selector-chain `wait` for a ready-screen landmark. +Duration waits, `wait stable`, and `wait @ref` do not qualify. +`wait stable` may follow the landmark wait. +Before filesystem work, the publisher validates the serialized guard. +It never relies on repair-only bare-ref rejection. +Without this **destination guard**, `session save-script` refuses and tells the author to record one. +V1 neither infers screen identity from snapshots nor synthesizes implicit guards. + +**Amendment (#1349).** The guard requires identity, not only a selector. +A selector wait recorded while ARMED captures landmark-mode `target-v1` evidence (ADR 0012's read-only-step amendment). +Only a selector wait with `verification: "verified"` qualifies. +An identity-empty element without id or label records no annotation and cannot qualify. +Publication refuses it with a recovery hint for a labeled/id-bearing landmark. + +Replay retains polling until a selector match has the recorded local identity and leaf-anchored ancestry prefix. +If only same-selector impostors exist at deadline, it fails closed with `identity-mismatch` `REPLAY_DIVERGENCE` before wait success. +A provider regression records, publishes, then replays against a rearranged tree whose same-label node has different id/ancestry. +It proves rejection of that false success. + +**Amendment (#1398, ADR 0017).** Echo-only identity also cannot qualify. +If a selector wait's only identity echoes a literal already parameterized through `fill --record-as`, it records no annotation. +Publication uses the same refusal hint and directs the author to a stable non-value-bearing landmark. +This enforces the manual correction in #1398's motivating scenario. + +The original selector-only guard proved existence, not landmark identity. +A rearranged screen could pass with the same weak label elsewhere. +[#1349](https://github.com/callstack/agent-device/issues/1349) owns identity design for waits and remaining read-only steps. +ADR 0012's pre-action verification cannot apply unchanged because an absent landmark is an expected initial wait condition. +The identity check therefore runs after selector resolution and before wait success. +ADR 0012's #1349 amendment defines the mechanism and read-only coverage classification. + +A request-sensitive recording-effect trait on central `CommandDescriptor` defines the last mutating action. +Every command that records session actions requires this trait under a completeness test. +The publisher has no local command-name set. +For `find`, `keyboard`, and `alert`, the trait distinguishes read-only from mutating subcommands. +Conservative `refFrameEffect: may-invalidate` is insufficiently precise here. + +Consumption preserves existing close-less replay behavior. +The named session remains active, and successful `ReplayCommandResult` returns its `session` id. +The caller binds subsequent commands to that id. +Replay reports success only after guard completion. +Absence of `close` changes neither action dispatch nor success timing. + +**Response shape (issue #1384).** `ReplayCommandResult` requires `sessionActive: boolean`. +Response construction computes it from the daemon store's current existence of `session`, never from script parse. +This enables the real CLI/IPC client to retain active handoff sessions. +Without it, one-shot `replay`/`test` teardown could not distinguish live handoff from completed work and stopped the owning daemon. +The client owner is `src/daemon-client/daemon-client-lifecycle.ts`. +`sessionActive` is true for every close-less run, including `--from` resume. +It becomes false after terminal `close` or the deferred equivalent in repair-armed runs. ### Sensitive inputs -Executable `.ad` artifacts serialize ordinary action inputs literally; diagnostic and event-log -redaction cannot protect an input that replay must later execute. ADR 0017 resolves #1348 for explicit -sensitive fills: `fill ... --record-as PASSWORD` sends the live text to the app while recording only -`${PASSWORD}`. Replay receives the value through `AD_VAR_PASSWORD` or `--env PASSWORD=`. +Executable `.ad` artifacts serialize ordinary action inputs literally. +Diagnostic/event-log redaction cannot protect inputs that replay must execute. +ADR 0017 resolves #1348 for explicit sensitive fills. +`fill ... --record-as PASSWORD` sends live text to the app but records only `${PASSWORD}`. +Replay receives the value through `AD_VAR_PASSWORD` or `--env PASSWORD=`. -This is opt-in and fill-only. Unparameterized fill/type inputs remain literal artifact content, so -authors must use ADR 0017 for each sensitive fill and avoid secret-bearing `type` steps. CLI help states -both the safe workflow and the remaining literal-input warning next to publication guidance. +This behavior is opt-in and fill-only. +Unparameterized fill/type inputs remain literal artifact content. +Authors must use ADR 0017 for each sensitive fill and avoid secret-bearing `type` steps. +CLI help places the safe workflow and literal-input warning beside publication guidance. -ADR 0017's #1398 amendment extends this protection to the whole recording session: a later, unrelated -recorded action can no longer re-serialize an app-rendered echo of an already-parameterized literal into -its own result or `target-v1` evidence, so a value protected once by `--record-as` stays protected for -every action recorded after it in the same session, not only its own originating fill. +ADR 0017's #1398 amendment protects the whole recording session. +Later unrelated actions cannot serialize an already-parameterized literal's app-rendered echo in their result or `target-v1` evidence. +A value protected once remains protected for all subsequent recorded actions in that session. ### Artifact contract The published `.ad`: -- contains exactly one recorded `open` as its first action and every recordable action through the - publication request; -- contains a portable selector/selector-chain destination guard after its last descriptor-classified - mutating action; -- does not append or serialize `session save-script` or `close`; -- uses the ordinary session context header, selector-chain optimization, and canonical `target-v1` - annotations captured while ARMED; -- fails loudly rather than emitting an unresolved session-local `@ref` or dropping target evidence that - ADR 0012 requires for an element-targeting recorded action; and -- uses the existing same-directory atomic publication primitive, refusing every existing target unless - `--force` authorizes atomic replacement. - -The success response identifies the final path and session and reports the number of serialized actions. -The command must fail before writing when there is no active session, recording was not armed, the -recording is ABORTED or PUBLISHED, the history does not contain exactly one initial `open`, no portable -destination guard exists, or a repair transaction owns the session. Every failure explains the recovery -action; none degrades to `{ written: false }` success. +- Contains exactly one initial recorded `open` and every recordable action through publication request. +- Contains a portable selector/selector-chain destination guard after its last descriptor-classified mutation. +- Appends neither `session save-script` nor `close`. +- Uses ordinary session context header, selector-chain optimization, and canonical `target-v1` annotations captured while ARMED. +- Fails explicitly on unresolved session-local `@ref` or missing target evidence required by ADR 0012. +- Uses the existing same-directory atomic publication primitive. + Every existing target is refused unless `--force` authorizes atomic replacement. + +Success identifies final path and session and reports serialized action count. +Before any write, the command must fail if: + +- No active session exists. +- Recording was not armed. +- Recording is ABORTED or PUBLISHED. +- History lacks exactly one initial `open`. +- No portable destination guard exists. +- A repair transaction owns the session. + +Every failure explains recovery. +None becomes `{ written: false }` success. ### Surface and naming -V1 extends the existing `session` command and typed session client surface. It does not introduce -`script start/stop`, marks, or a second replay engine. CLI help makes the two phases explicit: the -existing `--save-script` flag arms evidence capture, while `session save-script` publishes without -teardown. +V1 extends existing `session` and the typed session client surface. +It adds no `script start/stop`, marks, or second replay engine. +CLI help makes both phases explicit: + +1. `--save-script` arms evidence capture. +2. `session save-script` publishes without teardown. -This ADR does not rename or deprecate `--save-script`. The flag and session action name the same persisted -artifact: `open --save-script` configures the armed recording's eventual target, while -`session save-script` requests publication now. `save-replay` is rejected because replay is the act of -executing that script, not the artifact being saved. +This ADR neither renames nor deprecates `--save-script`. +The flag and session action identify the same artifact. +`open --save-script` configures the armed recording's eventual target. +`session save-script` requests immediate publication. +`save-replay` is rejected because replay executes the script instead of naming the saved artifact. ## Consequences -- Agents can record onboarding or deep navigation as one self-contained starting state, replay it from - scratch, and continue from the resulting live session. -- The workflow has two explicit moments because evidence must be armed before the first target action and - the destination is known only when the caller publishes. -- Normal unarmed interactions keep their current fast paths and retention behavior. -- A successful active-session publication cannot collide with a later close-time auto-save. -- A second successful open abandons the in-flight artifact instead of silently publishing a multi-open - bootstrap; authoring resumes only in a fresh session. -- Intermediate lifecycle-free fragments, entry guards, include semantics, composed digests, and shared - fragment pinning remain entirely under #1336. -- Secret-bearing fill authoring uses ADR 0017's explicit `--record-as ` contract. Unparameterized - fill/type inputs remain literal script content. Arbitrary history ranges remain out of scope. -- On consumption (issue #1384), a `replay` whose script has no terminal `close` leaves a live daemon and - app session behind — including the ordinary one-shot `replay` invocation's own owned/ephemeral daemon, - which the client keeps alive and reports a `--state-dir` address hint for instead of tearing down. An - unattended close-less replay (e.g. in CI) therefore leaks a session exactly like one opened - interactively: bounded by ordinary idle-reap or an explicit `close`, never by the one-shot command's own - process lifetime. This is the intended shape of "the caller owns close," not an accident, and recorded - test flows already end with `close` so `test` runs are unaffected. +- Agents can record onboarding or deep navigation as a self-contained starting state, replay it from the start, and continue in the live session. +- The workflow needs two explicit phases. + Evidence capture must start before the first target action. + The destination becomes known only at publication. +- Ordinary unarmed interactions retain fast paths and retention behavior. +- Successful active publication cannot conflict with later close-time auto-save. +- A second successful open abandons the current artifact instead of silently publishing a multi-open bootstrap. + Only a fresh session resumes authoring. +- Intermediate lifecycle-free fragments, entry guards, includes, composed digests, and shared fragment pins remain under #1336. +- Sensitive fills use ADR 0017 `--record-as `. + Unparameterized fill/type inputs remain literal script content. + Arbitrary history ranges remain outside scope. +- Close-less `replay` leaves a live daemon and app session (#1384). + This includes the ordinary one-shot invocation's owned/ephemeral daemon. + The client retains it and reports a `--state-dir` address hint. + Unattended close-less replay therefore retains a session like interactive open. + Ordinary idle reap or explicit `close` bounds its lifetime, not the one-shot process. + This intentionally assigns close ownership to the caller. + Recorded test flows already end with `close`, so `test` runs remain unaffected. ## Alternatives Considered -- **Save any session history at the end:** rejected because target identity evidence cannot be - reconstructed after the interaction and the resulting artifact would undercut ADR 0012's provenance - model. -- **Capture full target evidence in every session:** rejected until its direct-path latency, capture - count, memory, and event-log costs are measured. It would alter ordinary interaction behavior to make - one authoring command shorter. -- **`close --no-close` or `--save-script --no-close`:** rejected because a command named for teardown - would conditionally preserve the session and because it would not solve late arming. -- **General `script start/stop` or history marks:** rejected because the accepted v1 boundary is exactly - one recorded `open` through one destination. Arbitrary slices require entry-state semantics and belong - with fragment design. -- **`replay save`:** rejected because `replay ` consumes an artifact while publication consumes a - live session; the session owns the source data and lifecycle. -- **Infer a destination fingerprint at publication:** rejected for v1 because screen identity and - readiness are app semantics. A caller-authored target wait is explicit, already recordable, and fails - at the correct point during cold replay. +- **Save any session history at the end.** Rejected because target identity cannot be reconstructed after interaction. + The artifact would violate ADR 0012 provenance. +- **Capture full target evidence in every session.** Rejected until direct-path latency, capture count, memory, and event-log costs are measured. + It would change ordinary interaction behavior merely to shorten one authoring command. +- **`close --no-close` or `--save-script --no-close`.** Rejected because a teardown-named command would conditionally retain sessions. + It also would not solve late arming. +- **General `script start/stop` or history marks.** Rejected because v1 covers exactly one recorded `open` through one destination. + Arbitrary slices need entry-state semantics and belong to fragment design. +- **`replay save`.** Rejected because `replay ` consumes artifacts, while publication consumes live sessions. + The session owns source data and lifecycle. +- **Infer destination fingerprints at publication.** Rejected for v1 because screen identity and readiness are app semantics. + Caller-authored waits are explicit, recordable, and fail at the correct point in cold replay. ## Validation Required for Implementation -- An unarmed session refuses publication before filesystem work and names `open --save-script` as the - recovery. -- An armed session without a destination guard refuses publication before filesystem work and names a - selector-targeted `wait` as the recovery; `wait @ref`, duration waits, and `wait stable` are covered - refusal cases. -- A second successful plain `open` transitions ARMED to ABORTED and disables all publication, while - `open --save-script` on an existing session is rejected before app dispatch. -- An armed session publishes `open` plus target-annotated actions without `close`, returns the final path, - remains active, and can continue accepting commands. -- The artifact replays from a cold start, completes its destination guard, returns the live session id, - and accepts a subsequent command on that session. -- Every action in ADR 0012's existing target-binding command set has canonical identity evidence and no - unresolved `@ref` reaches disk; since #1349 the destination guard additionally requires verified - recorded landmark identity, with a reshuffled-screen false-pass regression proving replay fails - closed. -- Existing-target refusal preserves the original bytes; `--force` replaces atomically; a failed publish - remains retryable. -- After PUBLISHED, later ordinary actions remain usable, repeated `session save-script` fails, plain - `open --relaunch` cannot re-arm, and `open --save-script` is rejected before app dispatch. -- In ABORTED/PUBLISHED, `close --save-script[=]` is rejected before platform close and plain - `close` tears down without writing; closing an unpublished ARMED recording preserves current - close-time publication behavior. -- On a never-armed session (2026-08-02 amendment), `close --save-script` is likewise rejected before - platform close or filesystem work, naming `open --save-script` as the recovery; plain `close` still - tears down without writing, and the session is not deleted by the rejected request. -- Descriptor completeness tests classify every recordable request's mutation effect, including - request-sensitive read-only/mutating subcommands, and destination-guard ordering consumes only that - trait. -- Repair-armed sessions refuse this action without changing repair state. -- CLI help documents ADR 0017's `--record-as` workflow and warns that unparameterized fill/type inputs - remain literal script content. -- Provider-backed integration scenarios cover the public daemon route, and live iOS and Android runs - prove the saved artifact and post-save session behavior on real backends. +- Unarmed sessions refuse publication before filesystem work and name `open --save-script` as recovery. +- Armed sessions without destination guards refuse before filesystem work and name selector-targeted `wait` as recovery. + Tests include `wait @ref`, duration waits, and `wait stable` refusals. +- A second successful plain `open` changes ARMED to ABORTED and disables all publication. + Existing-session `open --save-script` rejects before app dispatch. +- Armed sessions publish `open` and annotated actions without `close`. + They return final path, remain active, and accept further commands. +- Cold replay completes the guard, returns the live session id, and accepts a later command in that session. +- Every action in ADR 0012's target-binding set has canonical identity evidence. + No unresolved `@ref` reaches disk. + Destination guards require verified landmark identity (#1349). + A rearranged-screen regression proves closed replay failure. +- Existing-target refusal preserves original bytes. + `--force` replaces atomically. + Failed publication remains retryable. +- After PUBLISHED, ordinary actions remain usable and repeated `session save-script` fails. + Plain `open --relaunch` cannot re-arm. + `open --save-script` rejects before app dispatch. +- In ABORTED/PUBLISHED, `close --save-script[=]` rejects before platform close. + Plain `close` tears down without a write. + Unpublished ARMED close preserves existing publication behavior. +- On never-armed sessions, `close --save-script` rejects before platform close or filesystem work and names `open --save-script` as recovery. + Plain `close` still tears down without a write. + The rejected request does not delete the session. +- Descriptor completeness tests classify every recordable request's mutation effect, including request-sensitive subcommands. + Only that trait controls destination-guard order. +- Repair-armed sessions refuse this action without repair-state changes. +- CLI help documents `--record-as` and literal unparameterized fill/type content. +- Provider-backed integration scenarios cover the public daemon route. + Live iOS and Android runs prove artifact contents and post-save session behavior on real backends. From 7b7b3c0758078c4596e531de403912fb8e7581b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:11:07 +0200 Subject: [PATCH 25/55] docs(adr): write ADR 0017 in ASD-STE100 --- .../adr/0017-parameterized-recorded-inputs.md | 291 +++++++++--------- 1 file changed, 150 insertions(+), 141 deletions(-) diff --git a/docs/adr/0017-parameterized-recorded-inputs.md b/docs/adr/0017-parameterized-recorded-inputs.md index 3daf3d3afd..c6fae53de1 100644 --- a/docs/adr/0017-parameterized-recorded-inputs.md +++ b/docs/adr/0017-parameterized-recorded-inputs.md @@ -4,22 +4,34 @@ Accepted +## Rules at a glance + +- Use fill-only `recordAs`, CLI `--record-as `, for explicit parameterized inputs. +- Send the literal to the app. Retain only `${VAR}` in the fill's response, recording, event, and script. +- Use uppercase `[A-Z_][A-Z0-9_]*` variable names. Reserve `AD_*` for runtime built-ins. +- Reject `--record-as` without armed recording or with `--no-record`. +- Never infer sensitivity or variable names from values, selectors, labels, or field names. +- Replay derives `recordAs` only for complete placeholder text tokens, preserving placeholders during recorded replay or repair. +- Protect later derived results and identity evidence from registered literal echoes under the session-scoped amendment's limits. +- Keep unparameterized fill/type inputs literal. Do not scan explicitly authored selectors or positional text. + ## Context -Ordinary `.ad` recording persists executable inputs. A literal `fill` value therefore becomes part of -the published script, including any password or token supplied during authoring. Diagnostic and event -redaction cannot make that executable artifact safe because replay still needs the value. +Ordinary `.ad` recording persists executable inputs. +A literal `fill` value therefore enters the script, including authoring passwords or tokens. +Diagnostic/event redaction cannot protect an executable artifact because replay still needs its value. Native replay already resolves `${VAR}` from `--env`/`-e` and `AD_VAR_*` immediately before dispatch. -The missing boundary is authoring: the live app needs the literal, while recording state and every -publication write must receive only the placeholder. Issue -[#1348](https://github.com/callstack/agent-device/issues/1348) defines this requirement; ADR 0016's -active-session publication deliberately excluded secret-bearing journeys until it was solved. +The missing boundary is authoring. +The app needs the literal, but recording state and every publication write must receive only the placeholder. +Issue [#1348](https://github.com/callstack/agent-device/issues/1348) defines this requirement. +ADR 0016 deliberately excluded secret-bearing journeys until its resolution. ## Decision -Add a fill-only `recordAs` input, spelled `--record-as ` on the CLI and exposed as `recordAs` by -the Node and MCP fill contracts: +Add fill-only `recordAs` input. +CLI spells it `--record-as `. +Node and MCP fill contracts expose `recordAs`: ```sh export AD_VAR_PASSWORD='' @@ -29,156 +41,153 @@ agent-device wait 'role="heading" label="Home"' agent-device session save-script ``` -The live fill dispatch receives the literal. Its public response, recording state, session event, and -published script use `${PASSWORD}`. Replay later resolves the placeholder from `AD_VAR_PASSWORD` or -`--env PASSWORD=` before the fill is dispatched. +Live fill dispatch receives the literal. +Its public response, recording state, session event, and script use `${PASSWORD}`. +Before replay dispatch, `AD_VAR_PASSWORD` or `--env PASSWORD=` supplies the value. -Variable names use replay's uppercase `[A-Z_][A-Z0-9_]*` grammar. `AD_*` remains reserved for runtime -built-ins. `--record-as` is rejected unless script recording is armed, and it cannot be combined with -`--no-record`. Ordinary fills without the option keep literal recording behavior. Sensitivity is never -inferred from a selector, label, field name, or value shape. +Variable names use replay's uppercase `[A-Z_][A-Z0-9_]*` grammar. +`AD_*` remains reserved for runtime built-ins. +Without armed recording, `--record-as` is rejected. +It cannot combine with `--no-record`. +Ordinary fills without it retain literal recording behavior. +Sensitivity is never inferred from selector, label, field name, or value shape. ### Mapping contract -The caller names the mapping explicitly. Reusing `PASSWORD` deterministically emits `${PASSWORD}`; -using `API_TOKEN` emits `${API_TOKEN}`. There is no secret-to-name table and no literal comparison or -deduplication state. This both avoids retaining a second copy of the secret and makes distinct-value -naming an authoring decision visible in the script. +The caller explicitly names the mapping. +Reused `PASSWORD` deterministically emits `${PASSWORD}`. +`API_TOKEN` emits `${API_TOKEN}`. +There is no secret-to-name table, literal comparison, or de-duplication state for naming. +This avoids a second retained secret copy for that purpose. +It also makes distinct-value names visible authoring decisions. ### Data-flow boundary The literal exists only on the live request path long enough to execute the interaction: -1. The request scope registers it as an explicit diagnostics secret before platform work. This is in - addition to key-name redaction, because an opaque value need not look secret. -2. The platform interaction receives the literal. -3. Before response/event/recording retention, the fill result's semantic `text` field becomes - `${VAR}` and every occurrence in a post-fill `refLabel` is parameterized. Stable selector/ref - provenance is preserved byte-for-byte. - Untrusted backend extras and nested settle output parameterize every occurrence of the supplied - literal in both keys and values; post-fill selector-chain candidates are dropped only when a - parsed `text`, `label`, or `value` term semantically contains it. Known response and settle - schema fields are the explicit structural boundary and retain their names and provenance values. - Because inline whitespace replacement cannot distinguish a sensitive value from ordinary - separators, an untrusted string or key containing a whitespace-only literal collapses to the - placeholder instead. -4. At `recordActionEntry`, the recorder reconstructs the fill's literal positional, writes only the - placeholder into `SessionAction`, and parameterizes the semantic result field plus arbitrary - backend/settle echoes again. ADR 0012 `target-v1` evidence keeps its structure; exact - value-bearing accessibility labels become the placeholder without rewriting identity fragments - that merely contain the same characters. The `recordAs` control flag itself is not serialized - into the script. Parameterization is idempotent: this second pass preserves placeholders inserted - at the response boundary even when the literal is part of the placeholder's variable name or is - `$`. -5. The existing ADR 0012/0016 writer receives an already-parameterized action. Its selector provenance, - portability checks, same-directory temp write, and atomic publication algorithm are unchanged. - -Because the target and temp file are written from the same prepared script string, neither can contain -the literal once this pre-publication boundary has run. +1. Before platform work, the request scope registers the literal as an explicit diagnostics secret. + This supplements key-name redaction because opaque values may not resemble secrets. +2. Platform interaction receives the literal. +3. Before response/event/recording retention, semantic result `text` becomes `${VAR}`. + Every occurrence in post-fill `refLabel` is parameterized. + Stable selector/ref provenance remains byte-for-byte unchanged. + Untrusted backend extras and nested settle output parameterize literal occurrences in keys and values. + Post-fill selector-chain candidates are dropped only if parsed `text`, `label`, or `value` terms semantically contain the literal. + Known response/settle schema fields define the structural boundary and retain names and provenance values. + For whitespace-only literals, an affected untrusted string or key collapses to the placeholder. + Inline replacement cannot distinguish sensitive whitespace from ordinary separators. +4. At `recordActionEntry`, the recorder reconstructs the fill's literal positional and writes only the placeholder into `SessionAction`. + It parameterizes semantic result and arbitrary backend/settle echoes again. + ADR 0012 `target-v1` evidence retains its structure. + Exact value-bearing accessibility labels become placeholders. + Identity fragments that merely contain the same characters remain unchanged. + The `recordAs` control flag is not serialized. + Parameterization is idempotent. + The second pass preserves existing placeholders even if the literal is `$` or part of the variable name. +5. The ADR 0012/0016 writer receives an already-parameterized action. + Selector provenance, portability checks, same-directory temp write, and atomic publication remain unchanged. + +Target and temp files use the same prepared script string. +After this pre-publication boundary, neither can contain the literal. ### Replay and repair -Missing variables retain replay's existing fail-loud behavior: `${VAR}` resolution throws before the -corresponding action is invoked. When replay dispatches an authored fill whose complete text token is a -`${VAR}` placeholder, its daemon-only provenance channel derives `recordAs=VAR` for that live request. -This derivation uses the unresolved source action, independently of whether the resolved value is empty -or whitespace. If the replay is itself being recorded or repaired, the recorder therefore retains the -original placeholder rather than serializing the expanded value. Embedded interpolation remains -ordinary script input; safe authoring emits one complete placeholder token. +Missing variables retain replay's explicit failure behavior. +`${VAR}` resolution throws before the corresponding action is invoked. +For authored fills whose complete text token is `${VAR}`, daemon-only provenance derives `recordAs=VAR` for the live request. +It uses unresolved source action, independently of empty or whitespace resolved values. +Thus, recorded replay or repair retains the original placeholder instead of the expanded value. +Embedded interpolation remains ordinary script input. +Safe authoring emits one complete placeholder token. ### Session-scoped echo protection (amendment, #1398) -> **Status: accepted.** Amends the data-flow boundary above from fill-step-scoped to -> recording-session-scoped, per issue [#1398](https://github.com/callstack/agent-device/issues/1398). - -The boundary above protects only the originating fill's own request/recording path. After -[#1349](https://github.com/callstack/agent-device/issues/1349), a later read-only action — -`wait`'s landmark-mode evidence, `is`, `get` — records `target-v1` identity evidence and a result -payload independently, computed from whatever the app renders at that later step. That capture has no -memory of an earlier fill's literal, so an app-rendered echo of it (the filled field's own displayed -value; a search result, validation message, or confirmation label containing it; a destination landmark -whose accessible label includes it) can re-enter session state and publication through a completely -different, unparameterized action — even though the originating fill was parameterized. - -**The guarantee is now recording-session-scoped, not fill-step-scoped.** For the lifetime of one -recording session, no later recorded action's own result payload or `target-v1`/`targets-v1` identity -evidence may re-serialize an app-rendered echo of a literal the session already parameterized. - -The mechanism is the smallest explicit, ephemeral state that can recognize a later echo: -`SessionState.recordedFillLiterals`, an in-memory `Map` populated only by the SAME -(literal, placeholder) pair a `fill --record-as` entry already computes for its own boundary above — one -entry per parameterized fill, added only after that fill's own entry has been recorded. It is never -serialized (not to the script, the session event log, or diagnostics), has no read API beyond the -recorder that owns it, and disappears with the session. - -This is not a reversal of the "no secret-to-name table" rule in the Mapping contract above. That rule -rejects retaining state to influence a *naming* decision — comparing a new `--record-as` value against -prior ones to infer or deduplicate a variable name. `recordedFillLiterals` never informs a naming -decision: every entry's placeholder is exactly the name the author already chose, and the map is -consulted only to recognize that same, already-named value reappearing in unrelated later evidence — a -redaction lookup, not a secret registry with a naming or comparison API. - -**Two treatments, by data class:** - -- **Result/event/backend-output fields** (display data, never compared at replay): content-aware - substring redaction, reusing the same recursive backend-output scrub the fill boundary already applies - to its own entry, generalized over every literal registered so far in the session. The substitution - itself is one placeholder-safe left-to-right pass over each string, not N sequential full-string passes - — a naive sequential pass over the same value can corrupt an EARLIER pair's just-inserted placeholder - when a LATER pair's literal happens to be a substring of it (register `somethinglong -> ${ABC}`, then - `ABC -> ${OTHER}`, and a second pass over a value already rewritten to `${ABC}` matches "ABC" *inside* - that token, producing `${${OTHER}}`). The single pass tries every registered literal longest-first at - each position — so one registered value that is a substring of another (a username that is a prefix of - a password) is never partially consumed by the shorter pair — and never re-scans text it has already - emitted, so no literal can ever be matched inside another pair's placeholder token in either direction. -- **`target-v1`/`targets-v1` identity evidence** (`label`, `ancestry[].label`, `scrollRegion.label` — the - fields replay's own classification compares): never silently text-substituted while still claiming a - trustworthy identity. Replay compares recorded identity against the *live* tree, which legitimately - re-renders the real value again at replay time; a placeholder written into a recorded identity field - could therefore never verify correctly. Instead: - - **Landmark mode (`wait`).** An echo is treated exactly like #1349's existing identity-empty case: no - annotation is recorded, and the wait keeps its selector-existence semantics. Because ADR 0016's - destination guard requires `verification: "verified"`, a landmark whose only identity is a - parameterized-value echo simply stops qualifying as a guard — `session save-script` refuses it with - the existing "record a selector-targeted wait on a labeled or id-bearing landmark" recovery, - reproducing the motivating scenario's real resolution (switching the guard to the stable `Apps` - landmark) as an enforced outcome rather than an authoring convention. - - **Action mode (`get`, `is`, mutating element-targeting actions).** ADR 0012/0016 require identity - evidence for every element-targeting recorded action and forbid silently dropping it, so an echo here - is never dropped. The literal-bearing label(s) are redacted to the placeholder (content-aware, - substring-based — unlike the exact-match fill-boundary redaction, because a cross-step echo is - typically surrounded by app-authored text such as "Welcome, ``") and `verification` is - downgraded to `"unverifiable"` — the same fail-closed downgrade decision 3's writer-parser invariant - already uses for an oversized payload. That fails the action's replay loudly - (`identity-unverifiable`) rather than silently weakening it to selector-only matching or publishing a - label that could never match again. +> **Status: accepted.** This amendment extends fill-step protection to recording-session scope under [#1398](https://github.com/callstack/agent-device/issues/1398). + +The original boundary protects only the originating fill request and recorded entry. +After [#1349](https://github.com/callstack/agent-device/issues/1349), later read-only actions independently record `target-v1` evidence and results from app-rendered state. +These include `wait` landmark evidence, `is`, and `get`. +Without memory of the earlier literal, an unrelated unparameterized action could retain an app-rendered echo through: + +- The filled field's displayed value. +- A search result. +- A validation message. +- A confirmation label. +- A destination landmark label. + +**The guarantee now applies to the recording session.** +During that session, no later action's result or `target-v1`/`targets-v1` evidence may serialize an app-rendered echo of an already-parameterized literal. +The explicit limits below apply. + +`SessionState.recordedFillLiterals` provides the minimum explicit ephemeral state needed to recognize later echoes. +This in-memory `Map` receives the same pair already computed by `fill --record-as` for its own boundary. +It adds one entry per parameterized fill, only after that fill's entry is recorded. +It never serializes to scripts, session event logs, or diagnostics. +Only its owning recorder can read it. +It disappears with the session. + +This does not reverse the Mapping contract's prohibition on a secret-to-name table. +That prohibition concerns state used to infer or de-duplicate variable names by comparison with earlier values. +`recordedFillLiterals` never selects names. +Every placeholder is exactly the author's chosen name. +The map only recognizes that named literal in later unrelated evidence. +It provides redaction, without a naming or comparison API. + +**Data classes receive two treatments:** + +- **Result/event/backend-output fields.** These display values are never compared during replay. + Content-aware substring redaction reuses the fill boundary's recursive backend-output scrub for every literal registered so far. + Each string receives one placeholder-safe left-to-right pass, not sequential full-string passes per literal. + Sequential passes could match a later literal inside an earlier inserted placeholder. + For example, register `somethinglong -> ${ABC}`, then `ABC -> ${OTHER}`. + Reprocessing `${ABC}` could produce `${${OTHER}}`. + Instead, the single pass tries registered literals longest-first at each position. + A shorter literal cannot partially consume a longer one, such as a username that prefixes a password. + The pass never scans emitted text again. + Thus, one pair's literal cannot match inside another pair's inserted placeholder in either direction. +- **`target-v1`/`targets-v1` identity evidence.** Replay compares `label`, `ancestry[].label`, and `scrollRegion.label` against the live tree. + These fields must never receive silent text substitution while retaining a trustworthy identity claim. + The live app legitimately renders the real value again. + A placeholder in trusted recorded identity therefore could not verify correctly. + Apply these rules instead: + - **Landmark mode (`wait`).** Treat an echo like #1349's identity-empty case. + Record no annotation and retain selector-existence semantics. + ADR 0016 requires `verification: "verified"` for destination guards. + A landmark whose only identity echoes a parameterized value therefore stops qualifying. + `session save-script` refuses it with existing "record a selector-targeted wait on a labeled or id-bearing landmark" recovery. + This enforces the motivating scenario's correction to stable `Apps` landmark selection. + - **Action mode (`get`, `is`, mutating element-targeting actions).** ADR 0012/0016 require identity evidence and forbid silent omission. + Never drop echo-bearing identity here. + Redact literal-bearing labels to placeholders through content-aware substring replacement. + Unlike the originating fill's exact-match rule, cross-step echoes commonly include surrounding app text, such as "Welcome, ``". + Downgrade `verification` to `"unverifiable"`, as decision 3's writer-parser invariant does for oversized payloads. + Replay then fails explicitly with `identity-unverifiable`. + It neither weakens to selector-only matches nor claims an identity label that cannot match again. **Explicit scope limits:** -- A whitespace-only or empty resolved fill value is excluded from the session-wide registry; it keeps - only the existing fill-step-scoped protection above. Collapsing arbitrary later strings on a value with - no discriminating content is a disproportionate readability cost for a value that reveals nothing - distinctive if echoed, and would corrupt unrelated short incidental substrings throughout the rest of - the recording. -- The originating fill's own recorded entry is protected only by the existing exact-match fill-boundary - pass; it is never passed through the coarser, substring-based session-wide pass using the pair it just - contributed. The session-wide guarantee is for later, distinct actions, exactly as #1398 frames it. -- An author's own explicitly typed selector or positional text is not scanned — only derived, resolved - evidence and result payloads are. An author who types the secret directly into a new selector is making - the same choice as an unparameterized fill/type value; that remains literal script content. -- The registry is keyed by literal, so two distinct `--record-as` names that happen to share the same - typed value (a password/confirm-password pair, say) are genuinely indistinguishable from a later - echo's perspective — the literal alone cannot say which fill produced it. The first-registered name - wins deterministically; the literal is redacted either way, so this only affects which placeholder - name a later echo is attributed to, never whether the value is protected. +- Empty or whitespace-only resolved values do not enter the session-wide registry. + They retain existing fill-step protection only. + They disclose no distinctive content through echoes. + Collapsing arbitrary later strings would impose disproportionate readability costs and corrupt incidental substrings. +- The originating fill's entry uses only its existing exact-match fill-boundary pass for the new pair. + It never uses that pair in the coarser session-wide substring pass. + The session-wide guarantee covers later distinct actions, as #1398 specifies. +- Explicitly authored selectors and positional text are not scanned. + Only derived resolved evidence and results receive protection. + A secret typed into a new selector remains literal script content, like unparameterized fill/type input. +- The registry key is the literal. + If distinct `--record-as` names share a typed value, later echoes cannot identify their source fill. + The first-registered name wins deterministically. + This changes placeholder attribution only, never literal protection. ## Consequences -- Secret-bearing login/bootstrap scripts can use ADR 0016 active publication safely. -- The caller must opt in for every sensitive fill; unparameterized fill/type values remain literal - script content and help warns accordingly. -- `type` and mutating `find ... fill|type` parameterization are not added by this decision. They require - their own surface and provenance design if needed. -- No new publication format, variable store, or sensitivity heuristic is introduced. The #1398 amendment - adds one ephemeral, non-serialized, per-session redaction map with no naming or lookup-by-name API — - never a persistent secret registry. +- Secret-bearing login/bootstrap scripts can safely use ADR 0016 active publication under these rules. +- Callers must opt in for every sensitive fill. + Unparameterized fill/type values remain literal content, with a help warning. +- This decision adds no parameterization for `type` or mutating `find ... fill|type`. + If needed, they require separate surface and provenance design. +- No new publication format, variable store, or sensitivity heuristic is introduced. + The #1398 amendment adds one ephemeral non-serialized per-session redaction map. + It has no naming or lookup-by-name API and never becomes a persistent secret registry. From 0e11c6764a9c5c45c075a4637ba5edf35a8acdd7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:11:58 +0200 Subject: [PATCH 26/55] docs(adr): repoint stale owner paths git log --follow identifies catalog move 2ec4e91b11 (R092) and errors move 76453add71 (R100). git grep confirms PUBLIC_COMMANDS and all four error exports at the new owners. Historical context and rejected alternatives keep their historical catalog path. --- docs/adr/0003-daemon-command-registry.md | 6 +++--- docs/adr/0010-error-system.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/adr/0003-daemon-command-registry.md b/docs/adr/0003-daemon-command-registry.md index c783754639..086e955ebc 100644 --- a/docs/adr/0003-daemon-command-registry.md +++ b/docs/adr/0003-daemon-command-registry.md @@ -6,7 +6,7 @@ Accepted ## Rules at a glance -- Keep public command identity in `src/command-catalog.ts` and public input/output contracts in `src/commands/**`. +- Keep public command identity in `packages/command-registry/src/catalog.ts` and public input/output contracts in `src/commands/**`. - Keep daemon route ownership and request-policy traits in the daemon command registry. - Request modules consume named predicates. Handler modules own execution logic without duplicate route coverage tables. - The daemon registry must not define public grammar, client options, schemas, help, or platform capability support. @@ -31,7 +31,7 @@ Duplicate command sets required alignment by convention. ## Decision -Keep public command identity in `src/command-catalog.ts`. +Keep public command identity in `packages/command-registry/src/catalog.ts`. Keep public input/output contracts in `src/commands/**`. Add `packages/daemon/src/core/daemon-command-registry.ts` as the daemon-side source of truth for command route ownership and request-policy traits. @@ -95,7 +95,7 @@ A derived daemon registry is therefore permitted **only if** it preserves all th 1. **Daemon-owned declaration.** A daemon-owned facet under `packages/daemon/src/` declares route and request-policy traits. The registration composes that facet. - The public command contract in `src/commands/**` or `src/command-catalog.ts` must never contain these traits as inline fields. + The public command contract in `src/commands/**` or `packages/command-registry/src/catalog.ts` must never contain these traits as inline fields. A registration call may be co-located with public declarations. Ownership of daemon policy must remain separate from the public surface. See Alternatives, “Keep daemon groups in `src/command-catalog.ts`”. diff --git a/docs/adr/0010-error-system.md b/docs/adr/0010-error-system.md index 1de13213d0..a5e4ef807e 100644 --- a/docs/adr/0010-error-system.md +++ b/docs/adr/0010-error-system.md @@ -19,7 +19,7 @@ ## Context -`src/kernel/errors.ts` centralizes `AppError`, `normalizeError`, `defaultHintForCode`, and `retriableForErrorCode`. +`packages/kernel/src/errors.ts` centralizes `AppError`, `normalizeError`, `defaultHintForCode`, and `retriableForErrorCode`. However, approximately 1,200 sites construct errors across CLI, daemon, and platform layers. A July 2026 audit combined code examination with iOS/Android use. It found a sound kernel but inconsistent call sites: From ca19d96b4bf33b6278e08e700d3d67adbcb524d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:11:58 +0200 Subject: [PATCH 27/55] docs(adr): write ADR index in ASD-STE100 --- docs/adr/README.md | 64 ++++++++++++++++++++++++---------------------- 1 file changed, 33 insertions(+), 31 deletions(-) diff --git a/docs/adr/README.md b/docs/adr/README.md index 1c56e3cb49..c4dd48c3de 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -2,52 +2,54 @@ | ADR | Read when you touch… | | --- | --- | -| [0001 Provider-First Integration Scenarios](0001-provider-first-integration-scenarios.md) | integration testing strategy, provider transcripts, the scenario harness | +| [0001 Provider-First Integration Scenarios](0001-provider-first-integration-scenarios.md) | integration test strategy, provider transcripts, the scenario harness | | [0002 Persistent Platform Helper Sessions](0002-persistent-platform-helper-sessions.md) | helper process lifecycle, keep-alive semantics | | [0003 Daemon Command Registry](0003-daemon-command-registry.md) | daemon routing, request-policy traits | | [0004 iOS Snapshot Backend Strategy](0004-ios-snapshot-backend-strategy.md) | snapshot capture plans, backend fallbacks, quality verdicts, the coordinate space a capture publishes | | [0005 iOS Runner Interaction Lifecycle](0005-ios-runner-interaction-lifecycle.md) | XCTest runner sessions, leases, adoption, idle-stop | | [0006 Daemon RPC Protocol Version](0006-daemon-rpc-protocol-version.md) | remote daemon HTTP/JSON-RPC compatibility | | [0007 Remote Device Leases](0007-remote-device-leases.md) | leases, tenancy, provider-owned devices | -| [0008 Command Descriptor Registry](0008-command-descriptor-registry.md) | adding/changing a command, any surface projection (CLI/MCP/client/batch), timeout policy | +| [0008 Command Descriptor Registry](0008-command-descriptor-registry.md) | command additions or changes, any surface projection (CLI/MCP/client/batch), timeout policy | | [0009 Apple Platform Consolidation](0009-apple-platform-consolidation.md) | Apple platform family, apple/appleOs axes, the apple-leak guard | | [0010 Error system conventions](0010-error-system.md) | error codes, hints, normalizeError, typed error signals | | [0011 Interaction Guarantee Contract](0011-interaction-guarantee-contract.md) | interaction dispatch paths, fast paths, guards, the guarantee matrix, parity tables | -| [0012 Interactive Replay](0012-interactive-replay.md) | replay healing/`--update`, diagnostic resolution disclosure, bounded `.ad` target-binding evidence, bounded divergence wire/error handling, plan-bound replay-only `--from` semantics, and agent-supervised re-record repair ("heal-by-doing") | -| [0013 Unified Gesture Plans](0013-unified-gesture-plans.md) | gesture API/routing, contact topology, multi-touch geometry, native pointer injection, two-finger pan | +| [0012 Interactive Replay](0012-interactive-replay.md) | replay repair/`--update`, diagnostic resolution disclosure, bounded `.ad` target-binding evidence, bounded divergence wire/errors, plan-bound replay-only `--from`, and agent-supervised re-record repair ("heal-by-doing") | +| [0013 Unified Gesture Plans](0013-unified-gesture-plans.md) | gesture API and routes, contact topology, multi-touch geometry, native pointer injection, two-finger pan | | [0014 Session Ref-Frame Lifetime](0014-session-ref-frame-lifetime.md) | ref authorization epochs, complete/partial issuance, pre-side-effect expiration, replay/batch compatibility, and cross-platform stale-mutation policy | -| [0015 Direct Maestro Compatibility Engine](0015-direct-maestro-engine.md) | Maestro YAML parsing/execution, compatibility observation policy, conformance, performance gates, gesture integration | -| [0016 Active-Session Script Publication](0016-active-session-script-publication.md) | publishing an armed open-to-destination `.ad` script without closing its live session | -| [0017 Parameterized Recorded Inputs](0017-parameterized-recorded-inputs.md) | safely authoring sensitive fill inputs as `${VAR}` placeholders across recording, replay, and repair | +| [0015 Direct Maestro Compatibility Engine](0015-direct-maestro-engine.md) | Maestro YAML parse and execution, compatibility observation policy, conformance, performance gates, gesture integration | +| [0016 Active-Session Script Publication](0016-active-session-script-publication.md) | publication of an armed open-to-destination `.ad` script that retains its live session | +| [0017 Parameterized Recorded Inputs](0017-parameterized-recorded-inputs.md) | safe authoring of sensitive fill inputs as `${VAR}` placeholders across recording, replay, and repair | | [0018 Unified Request Event Journal (Proposed)](0018-unified-event-journal.md) | event/diagnostic vocabulary, journal scopes and sinks, progress-channel separation, observability-only state | | [0019 Request-Bound Platform Runtime](0019-request-bound-platform-runtime.md) | platform-package boundaries/composition, device discovery, runtime facts/facets, request binding, provider ownership, platform-shaped session resources, durable reattachment, daemon-handler migration | | [0020 Composable Recorded Fragments (Proposed)](0020-composable-recorded-fragments.md) | lifecycle-free recorded fragment capture/composition, entry guards, fragment-local addresses/digests, staleness, and native `.ad`/Maestro composition | -| [0021 Simlock-Leased Devices Through Ordinary Local Runtimes](0021-host-simlock-managed-device-allocation.md) | driving a Simlock-leased simulator or emulator, why agent-device has no allocator integration, and the withdrawn in-process Host design | -| [0022 Daemon — Platform Runtime Coupling Audit and Ownership Ratchets](0022-daemon-platform-runtime-coupling.md) | daemon imports of root `platform-runtime-*` modules, the retired R76 edge classification now collapsed to a coupling ban, R75 session-authority ratchet, entry-to-platform hop routes and roles | -| [0023 End-State Entry-to-Platform Hop Trace](0023-end-state-hop-trace.md) | the hop counting definition, the ordered `press`/Android and dual-arm `snapshot`/iOS chains, per-hop roles, and the deletion test behind the numbers ADR 0019 and ADR 0022 quote | -| [0024 Screen Recording — One Coordinator, Two Facts (Proposed)](0024-screen-recording-provable-signal.md) | `record start`/`stop` lifecycle, the export-exists / recorder-stopped split, backend stop-and-collect contract, recovery through the coordinator, the ADR 0019 §5 amendment and the failed-finish cleanup fix | -| [0025 Foldable Apple Panels — Capture the Lit Panel](0025-foldable-apple-panels.md) | iPhone Duo outer/inner panels, CoreDevice display info as the only screen authority, why `simctl`'s implicit display default captures a black panel, derived (never asserted) pose, the `fold` pose control, and the Android emulator posture amendment | -| [0026 Scroll Clip Authority — Ownership Is the Parent Edge](0026-scroll-clip-authority.md) | iOS scroll indicators and the visible band, why ownership is read from the parent edge instead of an ancestor walk, what may remove a node from an interactive snapshot, and ejection dispositions | -| [0027 Descriptor Root Size vs Eager-Closure Budget](0027-descriptor-root-vs-eager-closure-budget.md) | accepted: the eager-closure budget tolerates a statement-neutral hub split only (containment + added-module novelty + flat top-level-statement weight) and nothing else; splitting `packages/command-registry/src/registry.ts` is unblocked | -| [0028 Capability-Family Cell Vocabulary — One Runtime Source (Proposed)](0028-capability-family-cell-vocabulary.md) | adding a capability operation family, `UnavailablePlatformRuntimeFacts` / `UNAVAILABLE_CELLS`, `INTERACTOR_OPERATIONS`, and why an eight-package fan-out recurs | -| [0029 Daemon Policy](0029-daemon-policy.md) | `AGENT_DEVICE_DAEMON_POLICY`, confining a daemon's commands, devices, or device shutdown, and where operator rules are enforced for batch/replay steps | +| [0021 Simlock-Leased Devices Through Ordinary Local Runtimes](0021-host-simlock-managed-device-allocation.md) | operation of a Simlock-leased simulator or emulator, the reason for no agent-device allocator integration, and the withdrawn in-process Host design | +| [0022 Daemon — Platform Runtime Coupling Audit and Ownership Ratchets](0022-daemon-platform-runtime-coupling.md) | daemon imports of root `platform-runtime-*` modules, the coupling ban that replaces R76 edge classification, the R75 session-authority ratchet, entry-to-platform hop routes and roles | +| [0023 End-State Entry-to-Platform Hop Trace](0023-end-state-hop-trace.md) | the definition of hop counts, ordered `press`/Android and dual-arm `snapshot`/iOS chains, per-hop roles, and the deletion test for numbers in ADR 0019 and ADR 0022 | +| [0024 Screen Recording — One Coordinator, Two Facts (Proposed)](0024-screen-recording-provable-signal.md) | `record start`/`stop` lifecycle, separate export-exists / recorder-stopped facts, backend stop-and-collect contract, coordinator recovery, the ADR 0019 §5 amendment, and cleanup after failed finish | +| [0025 Foldable Apple Panels — Capture the Lit Panel](0025-foldable-apple-panels.md) | iPhone Duo outer/inner panels, CoreDevice display info as sole screen authority, black-panel capture from `simctl`'s implicit display default, derived (never asserted) pose, `fold` pose control, and the Android emulator posture amendment | +| [0026 Scroll Clip Authority — Ownership Is the Parent Edge](0026-scroll-clip-authority.md) | iOS scroll indicators and the visible band, parent-edge ownership instead of ancestor traversal, permitted node removal from interactive snapshots, and ejection dispositions | +| [0027 Descriptor Root Size vs Eager-Closure Budget](0027-descriptor-root-vs-eager-closure-budget.md) | accepted: the eager-closure budget permits only a statement-neutral hub split (containment + added-module novelty + flat top-level-statement weight); `packages/command-registry/src/registry.ts` may be split | +| [0028 Capability-Family Cell Vocabulary — One Runtime Source (Proposed)](0028-capability-family-cell-vocabulary.md) | capability operation family additions, `UnavailablePlatformRuntimeFacts` / `UNAVAILABLE_CELLS`, `INTERACTOR_OPERATIONS`, and the reason for repeated changes across eight packages | +| [0029 Daemon Policy](0029-daemon-policy.md) | `AGENT_DEVICE_DAEMON_POLICY`, limits on daemon commands, devices, or device shutdown, and operator-rule enforcement for batch/replay steps | | [0030 Process Lock Exclusion](0030-process-lock-exclusion.md) | process-lock publication/reclaim/release, retained mutation guards, and the single-protocol upgrade boundary | -| [0031 macOS Native App Backend](0031-macos-native-app-backend.md) | `AGENT_DEVICE_MACOS_APP_BACKEND`, driving macOS app sessions without XCTest Automation Mode, why pointer actions are accessibility actions only, and the ghost cursor | -| [0032 Layering Graph Engine](0032-layering-graph-engine.md) | evaluating replacement of `scripts/layering/` graph rules with dependency-cruiser or fallow `boundaries`, why graph-based layering checks share one replaceable import model, the measured edge-set and parity gaps, and the revisit triggers | -| [0033 Collocation decision table](0033-collocation-decision-table.md) | moving or adding a file listed in umbrella #3276 §5, re-proposing a move that table refused, reading a zone's collocation decisions | +| [0031 macOS Native App Backend](0031-macos-native-app-backend.md) | `AGENT_DEVICE_MACOS_APP_BACKEND`, macOS app sessions without XCTest Automation Mode, the accessibility-only pointer action rule, and the ghost cursor | +| [0032 Layering Graph Engine](0032-layering-graph-engine.md) | replacement evaluation for `scripts/layering/` graph rules with dependency-cruiser or fallow `boundaries`, the reason for a shared replaceable import model, measured edge-set and parity gaps, and reevaluation conditions | +| [0033 Collocation decision table](0033-collocation-decision-table.md) | moves or additions for files in umbrella #3276 §5, renewed proposals for refused moves, and a zone's collocation decisions | | [0034 Provider plugin registry](0034-provider-plugin-registry.md) | provider plugins, the `agent-device/plugins` contract, plugin sources and order, duplicate provider ids, plugin failure isolation, the daemon plugin inventory and `daemon status` | -ADRs record *why*; the registries and gates they describe are the living source of truth — when -prose and a registry disagree, the registry wins and the ADR needs a follow-up. +ADRs record rationale. +The registries and gates they describe are the current source of truth. +If prose and a registry disagree, the registry takes precedence. +The ADR then needs a follow-up. -Shape conventions, so consulting an ADR stays cheap: +Use these shape conventions to keep ADRs inexpensive to consult: -- **Normative rules first, terse.** Status, then a "Rules at a glance" summary a reader can stop - after; full contracts and rationale below it. -- **Rationale and refuted alternatives stay in the ADR** — they are what stops re-litigating - settled ideas — but below the fold. -- **Process history is deleted once complete, not archived in-file.** Migration plans, per-step - landing tables, and point-in-time status change-logs go to git history; the Status section keeps - one line saying so plus any accepted, still-relevant waiver or evidence gap. -- Once a rule is gate-enforced, the ADR keeps the why and points at the gate rather than restating - the rule's details. +- **Put normative rules first.** Put Status before a terse "Rules at a glance" summary. + The summary lets readers stop there. + Put full contracts and rationale below it. +- **Retain rationale and rejected alternatives.** They prevent repeated discussion of settled decisions. + Put them below the summary. +- **Process history is deleted once complete.** Keep completed migration plans, per-step implementation tables, and point-in-time status logs only in git history. + In Status, keep one line that states this deletion. + Retain accepted waivers and evidence gaps that still apply. +- If a gate enforces a rule, retain its rationale and a gate pointer instead of repeated rule details. From 418b4b002e7e33651de1c7d193ce28940264f9b5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:15:23 +0200 Subject: [PATCH 28/55] docs(adr): remove completed ADR 0001 process history --- docs/adr/0001-provider-first-integration-scenarios.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/adr/0001-provider-first-integration-scenarios.md b/docs/adr/0001-provider-first-integration-scenarios.md index ba509f1b77..4223121b3d 100644 --- a/docs/adr/0001-provider-first-integration-scenarios.md +++ b/docs/adr/0001-provider-first-integration-scenarios.md @@ -4,6 +4,8 @@ Accepted +Completed process history is in git history. + > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Provider-first scenario coverage and semantic provider operations remain accepted. > Request-bound runtime facets and explicit provider ownership supersede the monolithic `Interactor` and request callback-stack topology one command at a time. @@ -217,6 +219,5 @@ Unit tests remain for: - Provider scope routing. - Platform error boundaries. -The temporary migration roadmap is complete and was removed from the repository. Future work should follow current evidence from the progress script, failing tests, or concrete adapter needs. It should not follow a permanent refactor backlog. From 87994438bc18206dc4fdc51e8034844b43056a71 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:15:23 +0200 Subject: [PATCH 29/55] docs(adr): remove completed ADR 0003 process history The current DAEMON_COMMAND_DESCRIPTORS is derived by deriveDaemonCommandDescriptors(commandDescriptors). Its source comment records byte-equal migration coverage and deletion of the hand-authored operand. Retain all four derivation invariants. --- docs/adr/0003-daemon-command-registry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/0003-daemon-command-registry.md b/docs/adr/0003-daemon-command-registry.md index 086e955ebc..c17dbce3ec 100644 --- a/docs/adr/0003-daemon-command-registry.md +++ b/docs/adr/0003-daemon-command-registry.md @@ -4,6 +4,8 @@ Accepted +Completed process history is in git history. + ## Rules at a glance - Keep public command identity in `packages/command-registry/src/catalog.ts` and public input/output contracts in `src/commands/**`. @@ -112,5 +114,3 @@ A single flat public descriptor that exposes daemon fields in public views is ** This would combine daemon policy with a public command registry, which this ADR rejects. Compose facets owned by their domains. Derive the registry from those facets. -Until registry tests verify the implemented derivation, the manually authored `packages/daemon/src/core/daemon-command-registry.ts` remains the source of truth. -The operating rule in `AGENTS.md` remains unchanged until then. From 35ffc5cdc321e257d85931d7453dbf9e8fde75ab Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:15:23 +0200 Subject: [PATCH 30/55] docs(adr): remove completed ADR 0014 facet migration note packages/command-registry/src/registry.ts now declares the app-switcher daemon facet and refFrameEffect. Delete only its obsolete pre-migration status and plan; retain generic-leaf transitions and completeness requirements. --- docs/adr/0014-session-ref-frame-lifetime.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/adr/0014-session-ref-frame-lifetime.md b/docs/adr/0014-session-ref-frame-lifetime.md index 7802f0a5e4..b37b0cb691 100644 --- a/docs/adr/0014-session-ref-frame-lifetime.md +++ b/docs/adr/0014-session-ref-frame-lifetime.md @@ -293,9 +293,6 @@ A later paste/type action invalidates independently. Generic routes have no exception. `back`, `home`, `orientation`, `scroll`, `tv-remote`, and `app-switcher` must cross the transition in the generic daemon leaf when they act. -A direct writer currently projects `app-switcher` to the daemon. -Unlike generic-routed siblings, it omits an explicit daemon descriptor facet and uses the registry's generic fallback. -The migration adds that facet and effect classification without a specialized route. The completeness gate covers every daemon-projected command, including generic fallbacks. Thus, a missing facet cannot hide an unclassified mutation. Mutations from unrelated external tools remain outside this session guarantee. From 2d42ce6a8149a990c40804865e55ca2bc4a8c834 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 31/55] docs(adr): restore the frozen meaning in ADR 0001 Restore ADR 0001 meaning that the STE rewrite changed: the "seam" term that code uses, the scope and condition of the synchronous Apple helper rule (glance and body), "creates pressure to promote them", "is naming pressure", and the dropped "because" reasons for semantic macOS, Linux, app-log, and recording contracts. Restore "The trade-off is coarser failure localization", "not yet represented", and the coverage-run qualifier on low-coverage files only. --- ...01-provider-first-integration-scenarios.md | 192 +++++++++--------- 1 file changed, 99 insertions(+), 93 deletions(-) diff --git a/docs/adr/0001-provider-first-integration-scenarios.md b/docs/adr/0001-provider-first-integration-scenarios.md index 4223121b3d..ad71b6bc9b 100644 --- a/docs/adr/0001-provider-first-integration-scenarios.md +++ b/docs/adr/0001-provider-first-integration-scenarios.md @@ -7,22 +7,24 @@ Accepted Completed process history is in git history. > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Provider-first scenario coverage and semantic provider operations remain accepted. -> Request-bound runtime facets and explicit provider ownership supersede the monolithic `Interactor` and request callback-stack topology one command at a time. +> Request-bound runtime facets and explicit provider ownership supersede the monolithic `Interactor` and the request callback-stack topology one command at a time. ## Rules at a glance -- For commands not yet migrated under ADR 0019, keep the legacy `Interactor` interface. -- For new command migrations, use bound runtime facets from ADR 0019. -- In provider-backed integration tests, run the real daemon request path. Replace only providers. -- By default, match provider transcripts without order. Use ordered transcripts only to test order. -- Keep HTTP coverage in a narrow suite for transport, auth, and response finalization. -- Keep synchronous host-tool calls local until their paths have semantic provider contracts. -- Mock-heavy handler tests should be deleted only after equivalent integration scenarios exist. +- For commands not yet migrated under ADR 0019, keep `Interactor` as the legacy semantic interface. +- For new command cutovers, use the bound runtime facets of ADR 0019. +- Provider-backed integration tests run the real daemon request path and replace only the providers. +- By default, provider transcripts match calls as an unordered contract. + Use ordered transcripts only when order is the behavior under test. +- Keep HTTP coverage as a narrow contract suite for JSON-RPC transport, auth, and response finalization. +- Synchronous host-tool calls are intentionally outside the provider seam. + Before a remote or cloud provider can own the path of a synchronous Apple helper, you must convert the helper. +- Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenario coverage exists. ## Context The test suite had many mocked unit tests for daemon handlers and dispatch. -These tests cost much to maintain and omitted important behavior in: +These tests were expensive to maintain, and they skipped important behavior in these areas: - Request admission. - Locking. @@ -32,64 +34,66 @@ These tests cost much to maintain and omitted important behavior in: - Interactor resolution. - Command translation in platform modules. -Android already had an ADB provider boundary. +Android already had an ADB provider seam. Apple-family and Linux platform modules mostly called host tools or runner commands directly. -Thus, device-free integration tests required mocks above important platform behavior. +Thus, device-free integration tests were difficult unless the tests mocked too high in the stack. ## Decision For commands not yet migrated under ADR 0019, keep `Interactor` as the legacy semantic interface between dispatch and platform behavior. -For new command migrations, use bound runtime facets from ADR 0019. +New command cutovers use the bound runtime facets of ADR 0019. -For commands that still use the legacy adapter, use these request-scoped provider boundaries below platform modules: +For commands that still use the legacy adapter, use these request-scoped provider seams below platform modules: - Android ADB provider. - Apple tool provider with semantic `simctl`, `devicectl`, macOS helper, and macOS host subproviders. - Apple runner provider. - Linux tool provider with semantic subproviders for desktop lifecycle, accessibility, clipboard, and screenshots. - App-log provider for semantic start and stop of app log streams. -- Recording provider for semantic local and remote screen-recording starts that would otherwise use raw host processes. +- Recording provider for semantic local and remote screen-recording start operations that would otherwise be raw host processes. -When platform intent is sufficiently stable to name, provider contracts should expose semantic operations. +When the platform intent is stable enough to name, provider contracts should expose semantic operations. Android already does this for install, pull, and port-reverse behavior. -Apple tool execution has started this change through semantic `simctl`, `devicectl`, macOS helper, and macOS host runners. -For unclassified host-tool paths, generic command execution remains a local compatibility fallback. +Apple tool execution has started to move in that direction with semantic `simctl`, `devicectl`, macOS helper, and macOS host runners. +Generic command execution remains as a local compatibility fallback for host-tool paths that are not classified yet. The macOS host provider owns these operations: - App open. - App inventory. -- Clipboard access. -- Appearance control. - -Remote desktop providers should not need to infer these intents from `open`, `find`, `plutil`, `pbcopy`, `pbpaste`, or `osascript`. -Linux desktop lifecycle exposes semantic `openTarget` and `closeApp` operations. -Provider-backed integration covers these user workflows. -Remote desktop providers should not need to infer intent from `xdg-open`, binary launch, `wmctrl`, or `pkill`. - -Linux accessibility snapshots, clipboard access, screenshots, and input synthesis are also semantic. -Their stable contracts are: - -- Capture this surface. -- Read or write clipboard text. -- Write a screenshot artifact. -- Perform this user input. - -These contracts describe intent independently of `python3`/AT-SPI, `xclip`, `scrot`, `grim`, `xdotool`, or `ydotool`. -App-log stream start and stop are semantic observability behavior. -Thus, provider-backed integration injects an app-log provider instead of starting `log stream` or `logcat`. -iOS simulator screen recording uses a recording provider. -Its stable contract starts simulator recording for a specified device and output path. -The local implementation uses `xcrun simctl io recordVideo`. - -Provider-backed integration tests run the real daemon request path and replace only these providers. +- Clipboard. +- Appearance. + +The reason is that remote desktop providers should not need to infer these intents from `open`, `find`, `plutil`, `pbcopy`, `pbpaste`, or `osascript`. + +Linux desktop lifecycle exposes semantic `openTarget` and `closeApp` operations for two reasons: + +- Provider-backed integration covers these user workflows. +- Remote desktop providers should not need to infer intent from `xdg-open`, binary launch, `wmctrl`, or `pkill`. + +Linux accessibility snapshots, clipboard, screenshots, and input synthesis are also semantic, because their stable contracts are these: + +- "capture this surface" +- "read/write clipboard text" +- "write a screenshot artifact" +- "perform this user input" + +The local implementation details `python3`/AT-SPI, `xclip`, `scrot`, `grim`, `xdotool`, and `ydotool` are not the contracts. + +App-log stream start/stop is semantic observability behavior, not generic host process execution. +Thus, provider-backed integration injects an app-log provider, and it does not spawn `log stream` or `logcat`. + +iOS simulator screen recording now uses a recording provider, because the stable contract is "start simulator recording for this device and output path". +The local implementation detail `xcrun simctl io recordVideo` is not the contract. + +Provider-backed integration tests run the real daemon request path and replace only those providers. Tests may use provider transcripts for platform command contracts. Tests may use scenario transcripts for broader user workflows. -By default, provider transcripts match calls without order. +By default, provider transcripts match calls as an unordered contract. Use ordered transcripts only when order is the behavior under test. -Prefer an in-process harness with providers for broad integration scenarios. -The harness should invoke the daemon request handler directly without a TCP listener. +For broad scenarios, prefer an in-process provider-backed integration harness. +The harness should invoke the daemon request handler directly, without a bound TCP listener. It should preserve: - Admission. @@ -98,82 +102,84 @@ It should preserve: - Handler routing. - Dispatch. - Platform modules. -- Provider boundaries. +- Provider seams. Keep HTTP coverage as a narrow contract suite for JSON-RPC transport, auth, and response finalization. -Descriptors control request provider scope inside the request router layer. +Inside the request router layer, descriptors drive the request provider scope. Each provider entry declares: - How the provider applies to the resolved device. - How the provider installs its scoped provider. - Which request-local data the provider exposes to handlers. -If a request id exists, Apple runner providers use request-id scope. -Only unscoped local runner calls may use a fallback based on the device alone. +When a request id exists, Apple runner providers are request-id scoped. +A fallback by device alone is allowed only for unscoped local runner calls. -Synchronous host-tool calls are intentionally outside the provider boundary. -Each remaining synchronous Apple helper is local-only. -Before a remote or cloud provider can own that path, convert the helper. +Synchronous host-tool calls are intentionally not part of the provider seam. +Any remaining synchronous Apple helper is local-only. +Before a remote or cloud provider can own the path of such a helper, you must convert the helper. -Remaining generic Apple host-tool calls intentionally stay local-only unless a new adapter requires semantic operations: +Remaining generic Apple host-tool calls intentionally stay local-only, unless a new adapter creates pressure to promote them: -- Perf process discovery and samples: `mdfind`, `ps`. +- Perf process discovery and sampling: `mdfind`, `ps`. - Launch diagnostics: `plutil`, `otool`. -- Local runner product build and signature: `swift`, `codesign`, synchronous `plutil`. +- Local runner product build and signing: `swift`, `codesign`, synchronous `plutil`. - Local simulator UI launch: `open -a Simulator`. - Plist compatibility fallbacks. - Fallback implementations behind semantic macOS providers. -Provider-backed integration scenarios should avoid scripts of these host commands for user workflows. -If a user-facing workflow depends on one, consider a semantic provider method for that intent. +Provider-backed integration scenarios should avoid scripted calls to these host commands for user workflows. +A user-facing workflow that depends on one of these commands is naming pressure for a semantic provider method. ## Alternatives Considered -- **Mock handlers or `dispatchCommand`.** These tests cost less to write. - They omit request admission, locking, session state, and platform command translation, where the main test gaps occurred. -- **Put the boundary at `Interactor`.** This is simpler and more uniform. - It bypasses platform modules and would miss the iOS/Linux host-tool connections that motivated this change. -- **Start with a full semantic provider per platform operation.** This gives a cleaner final design. - Too many operations require names in one change. - The migration starts where contracts exist or tests show a need. +- **Mock handlers or `dispatchCommand`.** These tests are cheaper to write. + But they skip request admission, locking, session state, and platform command translation. + These areas were the main sources of test blind spots. +- **Put the seam at `Interactor`.** This is simpler and more uniform. + But it bypasses platform modules, and it would not catch the iOS/Linux host-tool wiring issues that motivated this change. +- **Start with a full semantic provider per platform operation.** This gives a cleaner end state. + But it has too much surface to name correctly in one pass. + The migration starts where contracts already exist or where tests create pressure. - **Run every provider-backed integration scenario through HTTP.** This gives maximum end-to-end coverage. - Most scenarios would require TCP setup, sandbox permissions, and transport timeouts even when they do not test transport behavior. + But most scenarios would pay for TCP setup, sandbox permissions, and transport timeouts, even when transport behavior is not under test. ## Consequences Integration tests continue to cover platform command translation without real devices. -The request router owns a provider registry boundary. -The registry keeps platform-specific provider applicability local. -It composes provider scopes linearly. +The request router owns a provider registry seam. +But platform-specific provider applicability remains localized in that registry. +The registry composes provider scopes linearly. Thus, a new platform does not require another chain of nested wrappers. -New remote or cloud adapters can implement neutral provider contracts without changes to daemon, dispatch, or session contracts. -Generic tool-provider fallbacks provide interim compatibility. +New remote or cloud-backed adapters can implement neutral provider contracts without changes to daemon, dispatch, or session contracts. +Generic tool-provider fallbacks are an interim compatibility layer. They are not the target contract for cloud adapters. -If an integration scenario still scripts raw host commands, reassess whether the platform intent is sufficiently stable for a semantic provider method. +If a provider-backed integration scenario still scripts raw host commands, that is a signal to reassess whether the platform intent is stable enough for a semantic provider method. -Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenarios exist. -Unit tests remain appropriate for pure logic, parser matrices, selector matching, capability maps, and edge/error cases that integration tests express poorly. +Mock-heavy handler unit tests should be deleted only after equivalent provider-backed integration scenario coverage exists. +Unit tests remain appropriate for pure logic, parser matrices, selector matching, capability maps, and edge/error cases that integration tests would express poorly. -A provider-backed integration scenario covers more of the real request path. -However, its failures may require more diagnosis than failures in narrow unit tests. -Scenario names and provider transcript entries should describe user workflows and real e2e examples, so failures identify useful actions. +The trade-off is coarser failure localization. +A provider-backed integration scenario catches more of the real request path. +But its failures may require more diagnosis than failures in a narrow unit test. +Scenario names and provider transcript entries should stay rooted in user workflows and real e2e examples, so that failures remain actionable. -Coverage is expected to improve over the old suite of handler unit tests. -However, the first migration does not meet the original 90% target. +Coverage is expected to improve over the old handler-heavy unit suite. +But the first migration does not meet the original 90% target. The current coverage denominator also excludes some entrypoint and configuration files. -Thus, coverage should indicate trends instead of proving that tests exercise every public surface. +Thus, coverage should be treated as a trend signal, not as proof that tests exercise every public surface. `pnpm test:integration:progress` generates operational metrics. -After integration tests, CI runs `pnpm test:integration:progress:check` to prevent silent regressions in: +After integration tests, CI runs `pnpm test:integration:progress:check`, so that these metrics cannot silently regress: - Public-command coverage. - Device-observable workflow flag coverage. - Public-flag classification. -After a coverage run, the script is the source of truth for: +The script is the source of truth for: - Provider-backed integration size. - Handler-unit size. @@ -182,9 +188,9 @@ After a coverage run, the script is the source of truth for: - Command-family ownership. - Device-observable workflow flag coverage. - Provider transcript pressure. -- Files with low coverage. +- Low-coverage files after a coverage run. -These flag categories stay in their owning unit/CLI suites: +These flags stay in their owning unit/CLI suites: - Config. - Remote transport. @@ -193,21 +199,21 @@ These flag categories stay in their owning unit/CLI suites: - Report-writing. - Boot-fallback. -The report lists them as explicit exclusions from the denominator. -Provider pressure distinguishes semantic Apple `simctl`/`devicectl`, macOS helper, and macOS host usage from generic Apple host-tool usage. -It also distinguishes semantic Linux desktop/accessibility/clipboard/screenshot/input usage from generic Linux tool usage. -Thus, the report shows remote-adapter pressure without classifying named subproviders as raw shell intent. +The report lists these flags as explicit exclusions, so they are not silently missing from the denominator. +Provider pressure separates semantic Apple `simctl`/`devicectl`, macOS helper, and macOS host usage from generic Apple host-tool usage. +It also separates semantic Linux desktop/accessibility/clipboard/screenshot/input usage from generic Linux tool usage. +Thus, remote-adapter pressure is visible, and named subproviders do not count as raw shell intent. -The progress CLI should remain a small report and check runner over a progress model. +The progress CLI should stay a thin report and check runner over a progress model. The model owns discovery, coverage classification, provider-pressure accounting, and check-failure derivation. The CLI owns Markdown output and process exit behavior. The progress script runs as a Node type-stripped TypeScript script. -When the metadata API exposes the required facet, the script should consume command metadata directly. +When the metadata API exposes the necessary facet, the script should consume command metadata directly. Source parsing inside the model is an implementation detail, not the desired long-term interface. -It should remain limited to facets absent from runtime metadata. -Maps from typed client method calls to command names are one example. +Source parsing should remain limited to facets that runtime metadata does not represent yet. +One example is the map from typed client method calls back to command names. -Every public command should have at least one provider-backed integration scenario through the request router and request-scoped provider boundaries. +Every public command should have at least one provider-backed integration scenario that runs through the request router and the request-scoped provider seams. Unit tests remain for: - Parser matrices. @@ -219,5 +225,5 @@ Unit tests remain for: - Provider scope routing. - Platform error boundaries. -Future work should follow current evidence from the progress script, failing tests, or concrete adapter needs. -It should not follow a permanent refactor backlog. +Future work should be justified from live pressure in the progress script, from failing tests, or from concrete adapter needs. +A standing refactor backlog is not a justification for future work. From b257cae9995cacc524628a3492f8afa066d96b38 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 32/55] docs(adr): restore the frozen meaning in ADR 0002 Restore the "amended 2026-07" date on the shared Android helper sentence. Restore the deleted history clause on the old separate touch helper; it is the only base text that shows that gestures no longer stop the snapshot session. Drop the Status history line. Drop the false "retry limits below" scope, restore "an order of magnitude" and "not data reuse". Fix glance rules that dropped "only", widened the gesture retry ban, or changed the cache condition. --- ...002-persistent-platform-helper-sessions.md | 44 +++++++++---------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/adr/0002-persistent-platform-helper-sessions.md b/docs/adr/0002-persistent-platform-helper-sessions.md index 8d51be0ef7..779a844512 100644 --- a/docs/adr/0002-persistent-platform-helper-sessions.md +++ b/docs/adr/0002-persistent-platform-helper-sessions.md @@ -4,21 +4,19 @@ Accepted -Completed process history is in git history. - > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Persistent protocols, invalidation, and explicitly safe one-shot fallback remain accepted. -> The daemon owns helper lifetime policy. +> In this ADR, "daemon-owned" means that the daemon owns helper lifetime policy. > Platform modules own process-lived helper managers and helper mechanics. > Request bindings attach to healthy helper generations but never own or stop them. ## Rules at a glance - When startup costs are high and an automation context is reusable, use persistent platform helper sessions. -- The daemon owns lifetime policy. Platform modules own helper-manager mechanics. -- Unless an explicit cache contract applies, helpers must execute each command against fresh platform state. -- Keep Android snapshots and touch commands in one bundled automation helper. -- For Android, retry one-shot only for idempotent reads after the failed session stops. -- For Android, do not retry gesture injection after a session failure. +- The daemon owns helper lifetime policy. Platform modules own helper-manager mechanics. +- Unless a separate cache contract has explicit invalidation, a helper must execute each command against fresh platform state. +- On Android, snapshot capture, gesture viewport resolution, and planned-touch injection share one bundled automation helper. +- On Android, after a failed session command, retry one-shot only for idempotent reads, and only after the failed session has stopped. +- On Android, return a session gesture failure directly. Do not retry it one-shot. - If the packaged Android helper is unavailable or invalid, agent-device must fail closed. ## Context @@ -62,14 +60,14 @@ Unless a separate cache contract defines explicit invalidation, the helper must Use this session pattern: -1. On the first command that benefits from reuse, start the session lazily. -2. Bind the session to a device identity and helper/runner identity. -3. Communicate through a small validated protocol with request ids and version metadata. -4. While the identity and protocol remain valid, reuse the session. -5. When an invalidation condition occurs, invalidate the session. -6. If reuse fails, use the existing one-shot path for the current command, subject to the retry limits below. -7. Make shutdown best effort. -8. Make stale sessions disposable. +- On the first command that benefits from reuse, start the session lazily. +- Bind the session to a device identity and helper/runner identity. +- Communicate through a small validated protocol with request ids and version metadata. +- While the identity and protocol remain valid, reuse the session. +- When an invalidation condition occurs, invalidate the session. +- If reuse fails, use the existing one-shot path as the fallback for the current command. +- Make shutdown best effort. +- Make stale sessions disposable. These conditions invalidate a session: @@ -82,18 +80,20 @@ These conditions invalidate a session: For Android snapshots, productize a persistent helper mode that retains `UiAutomation` and serves fresh snapshot requests over an `adb forward` socket. Do not add snapshot result caching in this first step. -Infrastructure reuse provides the first reliable improvement. +The first reliable improvement is infrastructure reuse, not data reuse. The current implementation retains the existing one-shot instrumentation helper as the fallback for startup, socket, protocol, and request failures. Both transports execute the same packaged helper contract. If that helper is unavailable or invalid, agent-device must fail closed. It must not substitute the legacy `adb uiautomator dump` snapshot engine. Android permits only one reliable instrumentation-owned `UiAutomation` context per device. -Thus, snapshot capture, gesture viewport resolution, and planned-touch injection share one bundled automation helper (issue #1275; consistent with ADR 0013). +Thus, snapshot capture, gesture viewport resolution, and planned-touch injection share one bundled automation helper (amended 2026-07, issue #1275, consistent with ADR 0013). A live persistent helper session executes touch commands directly over its session socket. Without a live persistent session, the same helper runs one-shot. +Before that amendment, touch synthesis shipped as a separate instrumentation helper. +Thus, the daemon had to stop the persistent snapshot session before every gesture, and the next snapshot restarted it lazily. After a failed session command, one-shot retry applies only to idempotent reads, such as viewport resolution. -The failed session must stop before this retry. +The retry runs only after the failed session has stopped. Gesture injection is not idempotent because events may already be partially injected. Thus, session gesture failures return directly without a one-shot retry. Helper reuse must never let process ownership cause interference between commands. @@ -113,8 +113,8 @@ Avoid platform-specific startup assumptions in command handlers. ## Alternatives Considered -- **Keep only one-shot helpers.** This is the simplest robust option. - Android measurements show approximately ten times slower steady-state snapshots without reuse. +- **Keep only one-shot helpers.** This option is the simplest, and it is robust. + However, Android measurements show that it loses an order of magnitude of steady-state snapshot performance. - **Cache snapshots in the daemon.** This accelerates repeated reads. Without a mutation generation contract, it is unsafe after mutations, animations, navigation, system dialogs, or app process changes. Cache infrastructure can be added later. @@ -148,7 +148,7 @@ These tests cover: - One-shot fallback. Observability should report whether a command used, started, reused, or invalidated a persistent session, or used a one-shot fallback. -This information helps diagnose CI failures and user bug reports when a fast path fails. +When a fast path fails, this information keeps CI results and user bug reports diagnosable. Persistent sessions should not make direct interactive commands unexpectedly slow. Use short connect/request timeouts for the persistent path. From ebe30c8eb547747ee2d5c69693a9577255c3bfde Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 33/55] docs(adr): restore the frozen meaning in ADR 0003 Restore the deleted interim-owner sentence and the AGENTS.md rule at the end of the 2026-06 update. Drop the Status history line, since no history stays deleted. Use "pin" for the registry tests, and restore the exact denied-surface list and the "only"/export scope in the glance rules. Leave owner drift (daemon-command-registry.ts vs registry.ts, catalog.ts) for a maintainer decision. --- docs/adr/0003-daemon-command-registry.md | 30 ++++++++++++++---------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/docs/adr/0003-daemon-command-registry.md b/docs/adr/0003-daemon-command-registry.md index c17dbce3ec..e909deb7e8 100644 --- a/docs/adr/0003-daemon-command-registry.md +++ b/docs/adr/0003-daemon-command-registry.md @@ -4,15 +4,14 @@ Accepted -Completed process history is in git history. - ## Rules at a glance - Keep public command identity in `packages/command-registry/src/catalog.ts` and public input/output contracts in `src/commands/**`. - Keep daemon route ownership and request-policy traits in the daemon command registry. -- Request modules consume named predicates. Handler modules own execution logic without duplicate route coverage tables. -- The daemon registry must not define public grammar, client options, schemas, help, or platform capability support. -- A derived registry is permitted only if daemon-owned declarations, predicate access, public isolation, and compile-time completeness remain intact. +- Request modules consume named predicates from the registry. They do not recreate command string sets. +- Handler modules own execution logic only. They do not export duplicate coverage tables to prove route membership. +- The daemon registry must not define CLI grammar, Node.js client options, MCP schemas, user-facing help, or platform capability support. +- A derived daemon registry is permitted only if it preserves all four invariants in the 2026-06 update. ## Context @@ -37,7 +36,8 @@ Keep public command identity in `packages/command-registry/src/catalog.ts`. Keep public input/output contracts in `src/commands/**`. Add `packages/daemon/src/core/daemon-command-registry.ts` as the daemon-side source of truth for command route ownership and request-policy traits. -Request modules consume predicate functions from the registry instead of duplicate command string sets. +Request modules consume predicate functions from the registry. +They do not recreate command string sets. Handler modules own execution logic only. They do not export duplicate coverage tables to prove route membership. @@ -65,12 +65,13 @@ The command contract, projection, help, and capability modules retain these resp ## Consequences When a daemon-handled command is added or moved, its route and request-policy traits require updates in the daemon command registry. -Registry tests fix the expected trait decisions. +The registry tests pin the trait decisions. Provider-backed integration scenarios verify important request-policy behavior through the real daemon request path. -The registry file intentionally concentrates the internal contract. +The registry file is intentionally a dense internal contract. Its interface should remain small. -Callers ask daemon-policy questions through named predicates instead of direct access to command sets. +Callers ask daemon-policy questions through named predicates. +They do not read or mutate command sets. `AGENTS.md` should contain only the operating rule and relevant file pointers for agents. This ADR owns the rationale. @@ -87,12 +88,13 @@ It derives these surfaces from that site: - Batch allowlist. - Daemon registry. -This removes differences between tables that currently require alignment by convention. +Its purpose is to remove the drift between these tables. +Convention keeps several of these surfaces aligned against that drift. **This ADR's decision remains accepted.** Its boundary concerns ownership and the predicate interface, not the physical file that contains a trait. “Separate source of truth” means separate ownership and exposure through named predicates. -A projected or derived table can preserve that property. +That property remains true for a projected or derived backing table. A derived daemon registry is therefore permitted **only if** it preserves all these invariants: 1. **Daemon-owned declaration.** A daemon-owned facet under `packages/daemon/src/` declares route and request-policy traits. @@ -104,13 +106,15 @@ A derived daemon registry is therefore permitted **only if** it preserves all th 2. **Predicate interface unchanged.** Consumers continue to use named predicates such as `getDaemonCommandRoute`, `isLeaseAdmissionExempt`, and `shouldLockSessionExecution`. Only the daemon registry exposes these predicates. Derivation changes table construction, not table access. -3. **No leakage into public projections.** Types must prevent catalog/CLI/MCP/help/capability projections from access to daemon-only traits. +3. **No leakage into public projections.** Types must prevent the catalog/CLI/MCP/help/capability projections from reading daemon-only traits. The daemon registry must still not define CLI grammar, Node.js options, MCP schemas, user-facing help, or capability support. 4. **One declaration per concern, enforced by types.** At the single registration site, a missing or duplicate daemon trait must cause a compile error. This replaces alignment by convention. - This structural improvement justifies derivation instead of a separate manually authored table. + This structural improvement justifies derivation instead of a separate hand-authored table. A single flat public descriptor that exposes daemon fields in public views is **not** permitted. This would combine daemon policy with a public command registry, which this ADR rejects. Compose facets owned by their domains. Derive the registry from those facets. +Until that derivation is implemented and the registry tests pin it, the hand-authored `packages/daemon/src/core/daemon-command-registry.ts` remains the source of truth. +Until then, the operating rule in `AGENTS.md` does not change. From bf24c6857305219560219fa7d215d04b0ae22ffd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 34/55] docs(adr): restore the frozen meaning in ADR 0004 Restore the Status fact that the public surface stays two modes and that host AX is an acquisition route. Restore the no-box condition on wire `hittable`, the Swift/TypeScript guard referents, the Simulator probe routing condition, and the layering gate's "only". Rewrite the glance rules from the body: fallback disables the bridge except capture-scoped failures; restore body scope for node-count, raw transport, and facet rules. Restore the technical term "ladder", the semantic layer ordinals, past-tense history facts, and dropped modals ("can", "never", "always", "for any node"). --- .../adr/0004-ios-snapshot-backend-strategy.md | 926 +++++++++--------- 1 file changed, 472 insertions(+), 454 deletions(-) diff --git a/docs/adr/0004-ios-snapshot-backend-strategy.md b/docs/adr/0004-ios-snapshot-backend-strategy.md index 60b53a981a..723a07e8b9 100644 --- a/docs/adr/0004-ios-snapshot-backend-strategy.md +++ b/docs/adr/0004-ios-snapshot-backend-strategy.md @@ -3,45 +3,48 @@ ## Status Accepted. +This ADR was amended after local iOS Simulator acquisition moved to the host AX bridge. +The public surface remains two modes: regular interactive snapshots and raw diagnostic snapshots. +The host AX bridge is an acquisition route, not a third public mode. Completed process history is in git history. -The Apple platform runtime owns acquisition routes and its generation-scoped XCTest fallback. +The Apple platform runtime owns acquisition routing and its generation-scoped XCTest fallback. `@agent-device/capture-kit` owns host-side iOS validation, semantic presentation, and publication. -Structured snapshot quality verdicts and fallback warnings expose degraded or recovered output through the complete path. +Structured snapshot quality verdicts and fallback warnings make degraded or recovered output observable end to end. ## Rules at a glance -- For eligible local iOS Simulator snapshots, use host AX acquisition and shared TypeScript presentation. +- Route eligible local iOS Simulator snapshots through the host AX bridge. Present them once through the shared TypeScript engine. - If bridge acquisition or presentation fails, use one typed XCTest fallback. -- After an app-specific failure, disable the bridge for that app generation. A new generation enables it again. -- Keep screen-specific coordinate-space refusals scoped to the capture. -- Keep regular visibility and raw diagnostics as explicit public strategies. -- Regular snapshots must not omit visible controls through an arbitrary node-count cutoff. -- If raw output exceeds transport capacity, fail explicitly instead of silently truncating it. -- New host-side consumers must use the snapshot policy facet. -- Acquire raw facts. Apply visibility, eligibility, scope, and regular depth in presentation. -- Observe registered system surfaces in place. Never activate them. +- After fallback, disable the bridge for that app generation, except for capture-scoped failures. A new app generation enables it again. +- A refusal that provides evidence about the screen, not the app, has capture scope. +- Keep the regular visible strategy and the raw diagnostic strategy explicit as the two public snapshot strategies. +- The regular visible strategy must not use an arbitrary node-count cutoff. +- If the transport cannot carry a raw response, fail explicitly instead of silently truncating the tree at a hard node count. +- New consumers must use the host-side snapshot policy facet in `@agent-device/capture-kit` instead of adding another daemon presentation path. +- Acquisition backends serialize facts. Presentation applies visibility, eligibility, scope, and the regular depth cut. +- Observe and operate a registered system-surface host in place. Never activate it. ## Context -Agent Device exposes iOS UI state through host AX acquisition on local Simulators and the long-lived XCTest runner elsewhere. +Agent Device exposes iOS UI state through host AX acquisition on local Simulators and the long-lived XCTest runner everywhere else. The snapshot surface has two durable needs: -- Regular context for agents: effective user-visible UI, fixed controls such as tab bars, and hints about content outside visible scroll containers. -- Rich diagnostics and selector disambiguation: raw recursive XCTest snapshots that preserve hierarchy, static text, wrappers, scroll containers, and ancestry. +- Agent-facing regular context. Here the important contract is the effective user-visible UI, fixed controls such as tab bars, and scroll-hidden hints for content outside visible scroll containers. +- Rich diagnostics and selector disambiguation. Here a raw recursive XCTest snapshot is useful because it preserves hierarchy, static text, wrappers, scroll containers, and ancestry. -These needs should not use one capture strategy without distinction. -Recursive `XCUIElement.snapshot()` provides rich data. -However, some real app trees cause `kAXErrorIllegalArgument` or main-thread timeouts while the app remains visually usable. +These needs should not share one capture strategy without distinction. +Recursive `XCUIElement.snapshot()` is rich. +However, some real app trees can make XCTest fail with `kAXErrorIllegalArgument` or main-thread timeouts while the same app remains visually usable. Bluesky is the current known example. -Lower-level accessibility services can describe simulator screens even when recursive XCTest snapshots and typed `XCUIElementQuery` enumeration return no useful child nodes. -Physical iOS devices can have the same XCTest accessibility-channel timeout without an available lower-level semantic backend. +Lower-level accessibility services can describe simulator screens even when recursive XCTest snapshots and typed `XCUIElementQuery` enumeration degrade to no useful child nodes. +Physical iOS devices can show the same XCTest accessibility-channel timeout shape even when no lower-level semantic backend is available. -Presentation filters cannot correct this failure. +This failure is different from presentation filtering. The daemon's snapshot presentation can hide noisy or inaccessible nodes. It cannot recover nodes that XCTest never returns. -More filters, Maestro-specific heuristics, or daemon retries would only make the failure slower and less predictable. +More filters, Maestro-specific heuristics, or retries in the daemon would only make this failure slower and less predictable. ## Decision @@ -51,584 +54,599 @@ Present them once through the shared TypeScript engine. If bridge acquisition or presentation fails, use one typed XCTest fallback. After fallback, disable the bridge for that app generation, subject to the capture-scoped exception below. A new app generation enables it again. -Physical devices, providers, custom-action captures, and interactions retain their existing owners. - -A WebKit page in Safari or a `WKWebView` runs in a WebContent process. -Its connection to UIKit's tree is an `AXRemoteElement` under the web view. -Its children remain in the other process. -The bridge reads one process, so it returns the element as a leaf. -If that leaf has a `WebView`-typed ancestor and reaches the viewport, the source refuses the tree with `remote-content-boundary`. -Otherwise, published refs would target host views around the page instead of the page. -A zero-area or offscreen leaf cannot contain visible content omitted by the capture, so the source publishes it. -If the leaf reports no frame, the source refuses it because no fact proves it empty. - -A scene-hosted remote view controller, such as a share or action extension, uses the same structure. -It presents an `AXRemoteElement` leaf under a `_UISceneHostingView`. -The extension's controls remain in its own process. -The source refuses this leaf under the same rule. -Remote elements under other hosts remain unclassified because no capture has shown them. -If a node or depth cap removes the host, the capture continues to disclose truncation. -XCTest resolves remote elements, so the fallback serves the page or extension (#2484). - -This refusal opens the generation circuit because it provides evidence about the app. -After a hybrid app shows one web screen, its remaining native screens use XCTest until relaunch. -This is the 0.20.x path for every screen. -A bridge attempt on each capture would add a refused round trip to every `wait` poll on the web screen. -The circuit limits this cost to one capture per generation. -A screen-specific refusal instead has capture scope. -The coordinate-space amendment below defines that case. +Physical devices, providers, custom-action captures, and interactions remain on their existing owners. + +A WebKit page in Safari or in a `WKWebView` lives in a WebContent process. +It reaches UIKit's tree as an `AXRemoteElement` under the web view. +Its children are in that other process. +The bridge reads one process, so it delivers the element as a leaf. +If such a leaf has a `WebView`-typed ancestor and reaches the viewport, the source refuses the tree (`remote-content-boundary`). +The source refuses the tree instead of publishing a screen without its page. +If the source published that tree, refs issued from it would target the host views around the page instead of the page. +If the leaf frame is zero-area or off screen, the leaf hosts nothing that the capture can miss, so the source publishes it. +If the leaf reports no frame, the source refuses it because nothing proves it empty. + +A scene-hosted remote view controller (a share or action extension presented over the host app) reaches the tree in the same way. +It is an `AXRemoteElement` leaf under a `_UISceneHostingView`. +The extension's controls are in its own process. +The source refuses that leaf by the same rule. +Remote elements under any other host are not classified. +No capture has shown one. +If the node cap or the depth cap removes the host, the capture still discloses it as truncation. +XCTest resolves remote elements, so the fallback serves the page or the extension (#2484). + +The refusal opens the generation circuit, as any failure that provides evidence about the app itself does. +Thus, after a hybrid app shows one web screen, its remaining native screens use XCTest until the app relaunches. +In 0.20.x, every screen used this XCTest path. +A bridge request on each capture would instead add a refused bridge round trip to every `wait` poll on the web screen. +The circuit limits that cost to one capture per generation. +A refusal that provides evidence about the screen instead of the app has capture scope. +The coordinate-space amendment below decides that case. Keep the two public snapshot strategies explicit: - **Regular visible strategy**: use recursive XCTest snapshots. Emit the effective user-visible tree, visible ancestors, and scroll-hidden hints. - If XCTest returns sparse output, use the capture plan's fallbacks. - A node inside a scroll container is user-visible only if it intersects the app viewport and nearest visible scroll container. + If XCTest returns sparse output, fall back through the capture plan. + A node inside a scroll container is user-visible only if it intersects both the app viewport and the nearest visible scroll container. Offscreen descendants should be visited to set `hiddenContentAbove` / `hiddenContentBelow`. They should not be emitted as normal visible nodes. This strategy must not use an arbitrary node-count cutoff. - Fixed controls late in traversal, such as bottom tab bars after long lists, belong to the visible UI contract. -- **Raw diagnostic strategy**: use recursive XCTest snapshots for raw snapshots, diagnostics, and cases that require hierarchy. - Raw output may be noisy and large. - If the transport cannot carry the response, fail explicitly. - Do not silently truncate the tree at a hard node count. - If XCTest reports a real AX serialization failure, preserve the error instead of reporting an empty UI. + Fixed controls that are later in traversal order, such as bottom tab bars after long lists, are part of the visible UI contract. +- **Raw diagnostic strategy**: use recursive XCTest snapshots for raw snapshots, diagnostics, and cases that need hierarchy. + Raw output is allowed to be noisy and large. + If the transport cannot carry the response, fail explicitly instead of silently truncating the tree at a hard node count. + If XCTest reports a real AX serialization failure, preserve that error instead of reporting an empty UI. - **Host AX strategy**: acquire local Simulator trees as raw facts through the bounded host bridge. - Before publication, pass every result through the same presentation boundary. + Before publication, every result crosses the same presentation boundary. XCTest fallback carries explicit source residue. Comparisons require matching producer, intent, app generation, presentation key, and residue. Physical devices should use an equivalent non-XCTest semantic backend only if Apple exposes a supported channel. -The daemon should expose degraded output. +The daemon should make degraded output observable. If an iOS interactive snapshot contains only the application root or another sparse shape, return a structured quality verdict and warning. -Thus, agents can distinguish degraded output from proof that a screen has no controls. +Thus, agents know that the snapshot is degraded output, not proof that the screen has no controls. ## Host-side ownership boundary The shared TypeScript side has one snapshot-presentation facet. -`@agent-device/contracts/snapshot-presentation` owns the neutral acquisition-to-presented carrier and clip-fold geometry contract. +The neutral acquisition-to-presented carrier and the clip-fold geometry contract live in `@agent-device/contracts/snapshot-presentation`. `@agent-device/capture-kit` owns these host-side iOS responsibilities: -- Plans. -- Clip fold. +- Planning. +- Folding. - Projection. - Eligibility. - Semantic compaction. - Validation. - Publication. -Platform-specific presentation adapters retain only policy mechanics that cannot yet cross their runtime boundary. +Platform-specific presentation adapters retain only the policy mechanics that cannot yet cross their runtime boundary. Daemon assembly owns only the order of capture, compaction, occlusion, and ref publication. It does not own the presentation vocabulary or a second geometry carrier. Android acquisition remains in its platform module and adapts its raw hierarchy to the shared carrier. -Swift retains its runner-side `SnapshotPresentation` implementation because it consumes the capture-plan tier before the process boundary. -The iOS engine fixture provides shared proof between these runtimes. -It does not imply a shared implementation between Swift and TypeScript. +Swift keeps its runner-side `SnapshotPresentation` implementation because it consumes the capture-plan tier before the process boundary. +The iOS engine fixture is the shared proof between those runtimes. +It does not imply that Swift and TypeScript share an implementation. The macOS XCTest runner is the desktop-surface exception. Its already-presented nodes bypass the iOS presentation engine and continue through neutral snapshot assembly. -The host-side facet in `@agent-device/capture-kit` owns snapshot policy generally, including the three remaining Wave 4 policies tracked by #1983: - -- **Freshness recovery.** `packages/capture-kit/src/snapshot/snapshot-freshness/` owns the freshness window, Android staleness classification, its thresholds, and the retry loop. - A classifier and retry schedule parameterize the loop. - Thus, the permitted backend delay after a real transition is a policy input, not a loop-owned constant. - The schedule states a duration budget. - The loop derives the deadline from the window's `markedAt`. - Thus, the budget starts at the action, and callers cannot supply an incorrect absolute instant. - `packages/daemon/src/core/session-snapshot-freshness.ts` retains only session-dependent operations. - It reads and retires the window on store-owned `SessionState` and selects the comparison baseline from snapshot lineage. +The same split holds for the three remaining Wave 4 policies tracked by #1983. +Thus, the host-side facet in `@agent-device/capture-kit` owns snapshot policy generally, not only presentation: + +- **Freshness recovery.** The freshness window, the Android staleness classification and its thresholds, and the retry loop live in `packages/capture-kit/src/snapshot/snapshot-freshness/`. + A classifier and a retry schedule parameterize the loop. + Thus, "how long may a backend lag behind a real transition" is a policy input, not a constant that the loop owns. + The schedule is stated as a duration budget. + The loop itself derives the deadline from the window's `markedAt`. + Thus, the budget is always spent from the action, and a caller has no absolute instant that it could get wrong. + `packages/daemon/src/core/session-snapshot-freshness.ts` keeps only the operations that need a session. + It reads and retires the window on store-owned `SessionState`, and it chooses the comparison baseline from snapshot lineage. It remains the declared R7 owner of `androidSnapshotFreshness`. -- **Timeout evidence.** The deepest boundary with evidence classifies a hierarchy timeout once, from machine-defined values only. - `snapshot-capture-failure-reason.ts` maps the helper's structured `errorType` and SIGKILL exit code 137 to `accessibility-timeout`. - The `errorType` match requires exact equality with `java.util.concurrent.TimeoutException`. +- **Timeout evidence.** The deepest boundary that has the evidence decides once whether a capture failed because the hierarchy never arrived. + It decides from machine-defined values only. + `snapshot-capture-failure-reason.ts` maps the helper's structured `errorType` field and the SIGKILL exit code 137 to the typed reason `accessibility-timeout`. + The `errorType` match is an exact equality with `java.util.concurrent.TimeoutException`. `ANDROID_CAPTURE_FAILURE_REASONS` in `@agent-device/contracts/android-snapshot-quality` defines the typed reason. The helper-result, session-protocol, and killed-instrumentation error constructors attach it. - Each higher layer preserves it in its wrapper instead of reclassifying it. - No layer on this path examines message text. - Changes to helper or wrapper prose cannot change the reason. - Text that resembles a timeout does not create that reason. - End-to-end assertions against the real producer verify both properties. - `packages/capture-kit/src/snapshot/snapshot-timeout-policy.ts` reads the reason and derives the human-facing hint from it. - - `@agent-device/contracts/snapshot-timeout-evidence` owns the published `details.androidSnapshotTimeoutScreenshot` vocabulary. - Its union members encode which claims can coexist. - The annotated member carries a non-empty ref tuple, so callers cannot construct an annotation with zero refs. - No member stores a ref count. - A separate count would create a second source of truth that types cannot keep consistent. - Instead, the count is derived from the refs. - The daemon retains this required order: + Every layer above rewraps it instead of reclassifying it. + No code on that path examines a message shape. + Thus, a change to helper or wrapper prose cannot move the reason. + Prose that only resembles a timeout does not become one. + End-to-end assertions against the real producer verify both directions. + `packages/capture-kit/src/snapshot/snapshot-timeout-policy.ts` reads the reason. + The human-facing hint is derived from the reason, not decided beside it. + + The published `details.androidSnapshotTimeoutScreenshot` payload is vocabulary in `@agent-device/contracts/snapshot-timeout-evidence`. + It is a union whose arms encode which claims can coexist. + The annotated arm carries a non-empty ref tuple, so "annotated with zero refs" is not a state that a caller can build. + No arm stores a ref count. + A count beside the refs is a second source of truth that the type system cannot keep consistent. + Thus, the count is derived from the refs instead. + The daemon keeps the ordering that genuinely needs it: 1. Resolve a bound screenshot runtime. 2. Write the artifact. 3. Annotate it from the stored observation. 4. Emit the diagnostics. -- **Screenshot-overlay policy.** `packages/capture-kit/src/screenshot-overlay*.ts` decides which Android nodes receive an overlay ref and which rectangle the overlay covers. - The daemon retains only approved artifact and ref assembly: rank, projection to screenshot pixels, drawing, and PNG IO. +- **Screenshot-overlay policy.** `packages/capture-kit/src/screenshot-overlay*.ts` decides which Android nodes earn an overlay ref and which rectangle an overlay for one of them covers. + The daemon keeps only approved artifact and ref assembly: ranking, projection to screenshot pixels, drawing, and PNG IO. -`scripts/layering/snapshot-presentation-boundary.test.ts` enforces the dependency direction across roots declared by `snapshot-policy` in `scripts/layering/architecture-ownership.ts`. +`scripts/layering/snapshot-presentation-boundary.test.ts` enforces the direction, but only across the roots that `snapshot-policy` declares in `scripts/layering/architecture-ownership.ts`. Nothing in that snapshot tree may import `packages/daemon/src/`. -Overlay modules beside the tree are outside those roots and outside that gate. -The test has a positive control. -Without it, a filter that stopped matching would appear identical to a respected boundary. - -The residual call sites named by #1983 remain intentionally in place after an audit. -`packages/daemon/src/core/direct-ios-selector.ts` contains no presentation policy. -`isLocalIosRunnerSession` and `readSimpleIosSelectorTarget` route sessions by device family, provider ownership, and stabilization window. -`deriveDirectIosNodeSelector` and `isDirectIosSelectorFallbackError` provide selector derivation and ADR 0011 delegation-on-error. +The overlay modules beside the tree are outside those roots and outside that gate. +The test carries a positive control, because a filter that stopped matching would look identical to an obeyed boundary. + +The residual call sites that #1983 also named are audited and intentionally left in place. +`packages/daemon/src/core/direct-ios-selector.ts` carries no presentation policy. +`isLocalIosRunnerSession` and `readSimpleIosSelectorTarget` are session routing (device family, provider ownership, the stabilization window). +`deriveDirectIosNodeSelector` and `isDirectIosSelectorFallbackError` are selector derivation and ADR 0011 delegation-on-error. The latter two are pure and daemon-independent. -However, R19's selector pipeline would own them, not this facet. -A move under ADR 0004 would extend this ADR beyond its boundary. -These observation and interaction consumers access freshness only through the facet or its session binding: +However, their owner would be the selector pipeline that R19 governs, not this facet. +A move under ADR 0004 would widen this ADR to a boundary that it does not decide. +These observation and interaction consumers now reach freshness only through the facet or its session binding: - `selector-capture-runtime.ts`. - `deferred-interaction-outcome.ts`. - `snapshot-capture.ts`. - `interaction-touch-android-freshness.ts`. -New consumers must use the facet instead of another daemon presentation path. +New consumers must use the facet instead of adding another daemon presentation path. The acquisition/presentation boundary has two explicit vocabularies. -An acquired input contains raw node evidence, capture hint, viewport, lineage, and residue. +An acquired input is raw node evidence with its capture hint, viewport, lineage, and residue. The host engine folds and projects that evidence. -A presented input contains the runner's primary payload and validation facts. +A presented input is the runner's primary payload plus validation facts. The host engine validates it and performs semantic compaction once. -Regular eligibility determines membership in regular presentation. +Regular eligibility decides which nodes belong in the regular presentation. Publication adds refs and emits only the primary payload. -An optional unscoped quality payload is validated as classification evidence and is never published. - -Semantic compaction may move an identifier but may not delete its identity. -A structural `Other` wrapper with only an identifier is suppressed in favor of its content. -The content receives the identifier through delegation. -If the wrapper has no content, no recipient exists. -Its suppression would delete the identifier from every canonical view while `is`, `get`, and `click` still resolve it from the same capture. -That deletion requires the node's own declared `hittable: false`. -Only this verdict states that the wrapper is inert. -A producer that reports no hittability declares nothing. +An optional unscoped quality payload is validated for classification evidence and is never published. + +Semantic compaction may move an identifier, but it may not delete one. +A structural `Other` wrapper that carries an identifier and nothing else is suppressed in favor of its content. +This suppression is a delegation: the identifier continues to exist in whatever the wrapper represented. +If the wrapper has no content, it has nothing to delegate to. +Thus, its suppression deletes the identifier from every canonical view while `is`, `get`, and `click` still resolve it from the same capture. +That deletion needs the node's own declared `hittable: false`. +This is the only verdict in the capture that says the wrapper is inert. +A producer that reports no hittability for any node declares nothing. An absent fact is not a negative answer (#2638). ## Regression Notes -PR #639 made XCTest AX serialization failures explicit instead of returning empty snapshots. -This diagnostic behavior is correct, but it exposes apps whose accessibility trees XCTest cannot serialize. -Recovery belongs in the regular visible capture plan. -Healthy apps retain the fast recursive tree path. -When only fallback tiers can provide visible controls, degraded app classes can return bounded output with explicit limits. - -Issue #1105 identifies another failure in the same app class. -On large or animated screens, recursive capture can take many seconds before failure instead of immediately returning `kAXErrorIllegalArgument`. -This can exceed the runner's main-thread watchdog and fill the main queue with retries. -The plan passes its shared deadline to the query-sweep and private-AX tiers. -After the budget expires, later recovery tiers stop. -A slow, timed-out, or watchdog-abandoned XCTest capture penalizes that bundle's XCTest accessibility channel for a bounded window. -Subsequent regular plans derive their next step from backend traits through `effectiveSnapshotCapturePlan`. -If a runnable non-XCTest backend exists, the plan uses that independent tier. -Otherwise, as on current physical iOS devices, it uses a short XCTest probe instead of the full tree slice. -Thus, healthy screens can recover without another long capture of the failing screen. -The raw diagnostic plan is exempt and retains tree-first error propagation. - -The query-sweep tier has 19 `allElementsBoundByIndex` reads. -Each can fail with `kAXErrorIllegalArgument` on the same app class. -XCTest records each as a test failure worded `Failed to resolve query: ...`. -An unmuted failure ends `testCommand` when its main-thread block returns, regardless of `continueAfterFailure`. -A runner exit also loses the per-bundle penalty and depth memory. -Thus, the runner mutes AX-server rejections in both XCTest fetch wordings. -Each bounded main-thread dispatch that exceeds its slice counts as main-thread occupancy. -If a viewport read takes too long, the plan skips the sweep instead of adding it to the queue. +PR #639 made XCTest AX serialization failures explicit instead of returning them as empty snapshots. +That diagnostic change was correct, but it exposed apps whose accessibility trees XCTest cannot serialize. + +Later work moved recovery into the regular visible capture plan. +Thus, healthy apps keep the fast recursive tree path. +When fallback tiers are the only available source of visible controls, degraded app classes can still return bounded, honest output. + +Issue #1105 showed a second failure shape on the same app class. +On heavy or animating screens, the recursive tree capture can work for many seconds before it fails, instead of failing fast with `kAXErrorIllegalArgument`. +This pushes the chained plan past the runner's main-thread watchdog and fills the main queue with retries. +The plan now carries its umbrella deadline into the query-sweep and private-AX tiers. +When the budget is spent, later ladder rungs stop. +A slow, timed-out, or watchdog-abandoned XCTest-backed capture penalizes the XCTest accessibility channel for that bundle for a bounded window. +Subsequent regular plans derive the next step from backend traits (`effectiveSnapshotCapturePlan`). +If a runnable non-XCTest backend exists, they defer to that independent tier. +If it does not exist, as on physical iOS devices today, they run a short XCTest probe instead of the full tree slice. +Thus, healthy screens can recover without another long capture of the hostile screen. +The raw diagnostic plan is exempt and keeps tree-first error propagation. + +A third shape followed on the same app class after the plan recovered reliably. +On that app class, each of the query-sweep tier's 19 `allElementsBoundByIndex` reads fails with `kAXErrorIllegalArgument`. +XCTest records every one as a test failure worded `Failed to resolve query: ...`. +Any recorded failure that the runner does not mute ends `testCommand` as soon as the main-thread block that recorded it returns, whatever `continueAfterFailure` says. +Thus, the runner exited after (or during) every hostile snapshot, and the per-bundle penalty and depth memory were lost with it. +The runner now mutes AX-server rejections in both XCTest fetch wordings. +Every bounded main-thread dispatch that outlives its slice counts as occupying the main thread. +Thus, if a viewport read takes too long, the plan skips the sweep instead of queueing it. ## Recovery conformance and depth hints -The host AX bridge and the XCTest runner's private AX bridge retain separate recovery implementations. -They use different native representations, recovery tiers, and completeness evidence for rejected deep requests. -`contracts/fixtures/ios-ax-recovery-conformance.json` is their shared executable recovery contract. +The host AX bridge and the XCTest runner's private AX bridge recover rejected deep requests with different native representations, ladders, and completeness evidence. +They stay separate implementations. +`contracts/fixtures/ios-ax-recovery-conformance.json` is their shared, executable recovery contract. Each producer replays every case through its own adapter. -The fixture records per-producer expectations and intentional differences. -Thus, changes to either path use the same synthetic native evidence. -Extract common executable policy only where the fixture proves equivalence. +The fixture records per-producer expectations and the intentional differences. +Thus, a change to either recovery path is measured against the same synthetic native world. +Common executable policy is extracted only where the fixture proves equivalence. A shared engine is not a goal. -The host source also retains a bounded accepted-depth hint per resolved target generation and producer. -The hint changes only the native levels requested by the first request. -Only a finished recovery that observed a rejection can establish it. -It expires by hinted-capture count, so ordinary screens probe the full depth again. +The host source also keeps a bounded accepted-depth hint per resolved target generation and producer. +The hint changes only the native levels that the first request asks for. +It is learned only from a finished recovery that observed a rejection. +It expires by hinted-capture count, so ordinary screens probe back to the full depth. It is never shared across apps, generations, or producers. The route's generation circuit remains the only lifecycle owner. ## Consequences -Regular snapshots remain appropriate for agents and Maestro compatibility because they describe what a user can currently perceive and operate. -Raw snapshots remain appropriate when hierarchy matters. -Both may still fail explicitly on trees that XCTest cannot capture. -This failure is useful because another identical recursive capture is unlikely to return a different tree. - -A future AX-service backend is the correct place to restore semantic coverage for apps such as Bluesky. -It should be a platform backend with its own lifecycle, protocol, normalization, timing metrics, and fallback rules. -It should not be another special case inside the XCTest runner. - -The acquisition/presentation boundary preserves behavior through types. -Acquisition backends construct `RawAXNode`. -Only `SnapshotPresentation` constructs `PresentedNode`. -Response payloads accept only presented nodes. -Each capture-plan backend returns `SnapshotAcquisition`. -The exhaustive backend switch routes through one `SnapshotPresentation.present` call. +Regular snapshots remain the right tool for agents and Maestro compatibility because they describe what a user can currently perceive and interact with. +Raw snapshots remain the right tool when hierarchy matters. +Both may still fail explicitly on XCTest-broken trees. +That failure is useful because another identical recursive capture is unlikely to reveal a different tree. + +A future AX-service backend is the correct place to regain Bluesky-class semantic coverage. +It should be added as a platform backend with its own lifecycle, protocol, normalization, timing metrics, and fallback rules. +It should not be added as another special case inside the XCTest runner. + +The acquire/present migration begins with a typed seam that preserves behavior: + +- Acquisition backends construct `RawAXNode`. +- `SnapshotPresentation` alone constructs `PresentedNode`. +- Response payloads accept only presented nodes. + +Its second step also preserves behavior. +It makes every capture-plan backend return `SnapshotAcquisition`. +It routes the exhaustive backend switch through one `SnapshotPresentation.present` call. `PresentationOptions` is the stable request-policy input to that boundary. -`SnapshotPresentation` makes regular eligibility backend-neutral. +The first semantic migration layer makes regular eligibility backend-neutral inside `SnapshotPresentation`. The top-level viewport carrier survives. Every other node needs an interactive accessibility type or a non-empty label, identifier, or value. -Hittability does not admit an otherwise ineligible node. +Hittability no longer admits an otherwise ineligible node. Raw membership remains unchanged. -This is runner eligibility, distinct from daemon publication membership. -Backend-neutral daemon compaction retains ownership of its declared noise suppressions. -If eligibility removes a structural wrapper, presentation attaches surviving descendants to the nearest surviving ancestor. -It normalizes their indexes and depths. +This is runner eligibility, not daemon publication membership. +Backend-blind daemon compaction retains ownership of its declared noise suppressions. +If eligibility removes a structural wrapper, presentation reparents its surviving descendants to the nearest surviving ancestor. +Presentation also normalizes their indexes and depths. -Scope is a presentation specification, not an acquisition or daemon-compaction policy. +The second semantic layer makes scope a presentation specification instead of an acquisition or daemon-compaction policy. A trimmed non-empty scope selects the first presentation-preorder match whose subtree contributes to the requested projection. -The match examines label, identifier, and value without case sensitivity. -The selected subtree receives a new root. +The match examines label, identifier, and value case-insensitively. +The selected subtree is re-rooted. Depth applies relative to that root. -If no match exists, publication returns an empty healthy projection. -`contracts/fixtures/snapshot-scope-policy.json` fixes expectations for Swift and TypeScript implementations. -Scoped iOS acquisition remains broad, including with requested depth, until an adapter can prove a narrowing hint complete. +If no node matches, presentation publishes an empty healthy projection. +`contracts/fixtures/snapshot-scope-policy.json` pins the Swift and TypeScript implementations. +Scoped iOS acquisition stays broad (including when depth is requested) until an adapter can prove a narrowing hint complete. The daemon never reapplies scope after the wire. -Android selects its root inside TypeScript presentation. +Android selects its root inside its TypeScript presentation. Desktop surface runtimes retain their platform projection. -Presentation has two projections, and acquisition has one input. +The third semantic layer splits presentation into two projections and gives acquisition one input. `SnapshotPresentation.captureHint` derives a `CaptureHint` from the request. Backends read the hint, never `PresentationOptions`. -A hint names the projection that acquisition must serve. -It may narrow acquisition only where the backend can prove completeness for that projection. +A hint names the projection that the acquisition must serve. +A hint may narrow acquisition only where the backend can prove the narrowing complete for that projection. Scope and its relative depth never narrow acquisition. -Raw depth does narrow acquisition because it is traversal depth. +Raw depth does narrow it, because raw depth *is* traversal depth. The raw projection never carries `interactiveOnly`. `presentRegular` folds visibility, eligibility, scope, and scroll hints. -`presentRaw` returns the normalized acquired tree. -It applies scope and depth only when the request specifies them. -Thus, `interactive ⊆ regular ⊆ raw` holds per backend, independently of implementation. -`snapshot --raw -i` therefore returns the acquired tree without an interactive filter. - -Two structural rules prevent a backend from returning the wrong projection. -Without them, a recovered `snapshot --raw` could return viewport-pruned nodes labeled raw. -The raw diagnostic plan derives from `SnapshotBackendKind.supportsRawProjection`, not a manual list. -Thus, a backend without hierarchy, such as query sweep, cannot enter a raw plan. -Presentation compares the requested projection with the acquisition's capture hint. -If they differ, it drops that tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure. +`presentRaw` is the acquired tree, normalized. +It applies scope and depth only when the request asked for them. +Thus, `interactive ⊆ regular ⊆ raw` holds per backend, not per backend implementation. +`snapshot --raw -i` therefore returns the acquired tree instead of an interactive-filtered tree. + +Two structural rules keep a backend from answering a request with the other projection. +An answer with the other projection is the shape that let a recovered `snapshot --raw` return viewport-pruned nodes labeled raw. +The first rule derives the raw diagnostic plan from `SnapshotBackendKind.supportsRawProjection`, not from a manual list. +Thus, a backend with no hierarchy to return (the query sweep) cannot be planned for raw. +In the second rule, presentation compares the requested projection with the hint that the acquisition was captured under. +If they differ, presentation drops that tier with a structured `IOS_SNAPSHOT_PROJECTION_MISMATCH` failure. It does not present that tier under the requested label. -The clip fold belongs to presentation. -Acquisition backends serialize facts. -They emit each traversed node at raw traversal depth with its reported frame. +The fourth semantic layer moves the clip fold itself into presentation. +Acquisition backends are fact serializers. +They emit every traversed node at raw traversal depth with its reported frame. `SnapshotAcquisition` carries the viewport. -The fold returns a typed carrier with reported and effective geometry. +The fold returns a typed carrier with both values: reported geometry and effective geometry. `raw.rect` remains runner-internal reported geometry. Regular presentation writes the carrier's effective rectangle through the existing wire `rect` field. Raw projections and direct single-element reads retain reported geometry. -For every backend, `presentRegular` runs one visibility interpreter with: +For every backend, `presentRegular` runs the one visibility interpreter, which includes: - Viewport ∩ scroll-container clip. -- An ancestor projection cursor. +- The ancestor projection cursor. - The sub-pixel decoration rule. -- Hidden-content hints on scroll anchors. -- Surviving nodes attached to surviving ancestors with collapsed depth. - -The cursor prevents an out-of-clip Cell or scroll container's descendants from reappearing through clamped frames. -The fold also restricts emitted `hittable` to the clip. -Nodes outside their clip or without geometry are never hittable, regardless of backend reports. -Platform differences enter the shared algorithm through `SnapshotFoldPolicy`, never a backend exception. -iOS uses cursor projection. +- Hidden-content hints booked onto scroll anchors. +- Reparenting of survivors with collapsed depth. + +With the cursor, an out-of-clip Cell or scroll container hides descendants whose clamped frames would otherwise leak back into the viewport. +The fold also narrows the emitted `hittable` to the clip. +Nothing outside its clip, and nothing without geometry, is ever hittable, regardless of what a backend reported. +Platform differences are a `SnapshotFoldPolicy` input to the shared algorithm, never a backend exception. +iOS is cursor-projected. macOS/tvOS use plain viewport intersection. -Before constructing `SnapshotPresentation.PresentedNode`, the presentation owner validates each framed regular node against its cumulative effective clip. -Frameless and degenerate semantic carriers remain eligible but never actionable. +Before it constructs `SnapshotPresentation.PresentedNode`, the presentation owner validates every framed regular node against its cumulative effective clip. +Frameless and degenerate semantic carriers stay eligible but are never actionable. Raw projection remains exempt by contract. -A violation produces a typed `IOS_SNAPSHOT_PRESENTATION_FAILED` capture failure with the named `presentation-failed` snapshot-quality reason. +A violation is a typed `IOS_SNAPSHOT_PRESENTATION_FAILED` capture failure with the named `presentation-failed` snapshot-quality reason. Recovery and the existing TypeScript verdict/warning contract preserve that failure. -Inside the runner, `SnapshotViewport` is a declared fact with the same cases as host `IosViewportEvidence` (#2891): +Inside the runner, the viewport is a declared fact, not a rectangle. +`SnapshotViewport` has the cases of the host's `IosViewportEvidence` (#2891): - `reported(box, interfaceOrientation)`. - `derived(box)`. - `missing(reason)`. -Only `reported` carries an orientation, so only it can establish a rotation in `SnapshotGeometrySpace`. -Without a box, the clip is skipped, and the cumulative-clip invariant has no root clip to violate. -A node whose actionability depends on containment has no wire `hittable`, as on the host bridge. -Disabled or degenerate nodes remain explicitly `false`. -A rectangle becomes a `Box` only through the validating initializer. -The shared guard rejects `CGRect.infinite` by identity. -Its components and extents are finite, so numeric comparisons cannot reject the value left by a failed read. -Both guards reject that shape and finite components whose right or bottom edge overflows. -They preserve every other box classification that either guard previously accepted. -The runner route derives host evidence from payload root nodes through `resolveIosViewportEvidenceFromRoots` in `packages/capture-kit/src/ios-snapshot-acquisition.ts`. -`contracts/fixtures/snapshot-actionability-policy.json` fixes the predicate for shapes outside the 320x240 fold fixture's range. - -A regular `--depth` request cuts presentation, not acquisition. -`CaptureHint` separates raw traversal depth (`--raw --depth`) from regular presented depth. -During descent, the recursive tree walk reads neither presented depth nor geometry. -For regular capture, it enumerates the hierarchy under a runaway node cap (#1105, #1156). -It serializes each node at reported traversal depth with its platform-reported frame. -A single post-acquisition pass over the flat array decides coordinate space from ancestry. -`SnapshotGeometrySpace.normalized` runs in `captureWithBackend` (#2661). -It replaces the per-node `geometrySpace` / `parentIsWindow` values from #2612. - -A post-acquisition normalization pass cannot supply normalized geometry to an earlier traversal decision. -Thus, a visible-depth frontier that consults the shared fold during traversal would read reported geometry. -It could cut a rotated keyboard band in native device space at the wrong presented depth. -No acquisition-time decision therefore reads geometry. -Only raw traversal depth and the node cap bound the walk. -Inside `SnapshotPresentation`, visibility fold and presented-depth cut use the normalized array at `maximumDepth`. - -The earlier frontier applied only with presented depth, through `snapshot --depth N`. +Only `reported` carries an orientation, so only it can anchor a rotation in `SnapshotGeometrySpace`. +If the viewport has no box, the clip is skipped, and the cumulative-clip invariant has no root clip to violate. +If the viewport has no box, a node whose actionability depends on containment has no wire `hittable`, as on the host bridge. +Disabled or degenerate nodes stay declared `false`. +A rectangle becomes a `Box` only through the initializer that checks it. +The shared guard refuses `CGRect.infinite` by identity. +The components and the extents of that value are all finite, so no comparison would have caught the value that a failed read leaves behind. +The Swift guard and the TypeScript guard both refuse two shapes: `CGRect.infinite`, and finite components whose right or bottom edge overflows. +Every box that either guard accepted before #2908, and that is neither of those two shapes, is still classified the same way. +The runner route's host evidence comes from the payload's root nodes (`resolveIosViewportEvidenceFromRoots` in `packages/capture-kit/src/ios-snapshot-acquisition.ts`). +`contracts/fixtures/snapshot-actionability-policy.json` pins the predicate for shapes that the 320x240 fold fixture cannot reach. + +A regular `--depth` request is a presentation cut, not an acquisition bound. +`CaptureHint` keeps raw traversal depth (`--raw --depth`) separate from regular presented depth. +However, the recursive tree walk no longer reads presented depth (or any geometry) while it descends. +For a regular capture, it enumerates the hierarchy, and a runaway node cap bounds the walk (#1105, #1156). +It serializes each node at reported traversal depth with the frame that the platform reported. +The change in #2661 also removed the per-node coordinate space (`geometrySpace` / `parentIsWindow`) that #2612 had threaded through every walker. +The one coordinate-space decision is now a single post-acquisition pass over the flat array. +That pass (`SnapshotGeometrySpace.normalized`, run in `captureWithBackend`) keys on ancestry instead of a value carried down the stack. + +That pass is why the earlier visible-depth frontier was removed. +The frontier consulted the shared fold during the walk. +A post-walk normalization pass cannot feed a decision that the walk has already taken. +The fold would read reported geometry. +Thus, a turned keyboard band in the device's native space could be cut at the wrong presented depth before the pass ran. +With the frontier gone, no acquisition-time decision reads geometry. +The walk is bounded only by raw traversal depth and the node cap. +The visibility fold and the presented-depth cut both happen inside `SnapshotPresentation`, on the normalized array, at `maximumDepth`. + +The frontier only engaged when a presented depth was set (`snapshot --depth N`). `snapshot -i` carries no presented depth. -Thus, only the former route could change cost after its removal. -Measurements used the recursive tier, 15 warm captures per cell, and `main` on the same simulator: +Thus, `snapshot --depth N` is the route whose cost the change could move. +Measurements on that route used the recursive tier, 15 warm captures per cell, against `main` on the same simulator: - Form: 135 raw nodes, 4 or 6 presented. - Scrolled catalog list: 279 raw nodes, 4 or 6 presented. - Acquisition p50: within ±4 % of `main`, with mixed sign. - Acquisition p95: within ±7 % of `main`, with mixed sign. -The changed walk visits the full raw tree where `main` pruned at presented depth. -The platform already materializes the `XCUIElementSnapshot` tree in one call, which dominates acquisition time. -The saved node construction has no measurable cost on these trees. -Screens that recover to private AX remain unaffected. -The raw node cap remains the bound. -De-duplication drops a repeated node and attaches its children to that node's parent. -Thus, identical rows have one addressable owner instead of a subtree split across duplicate identities. +On the measured branch, both screens walk their whole raw tree, where `main` pruned it at the presented depth. +The walk runs over an `XCUIElementSnapshot` tree that the platform has already materialized in one call, which is where acquisition time goes. +The node construction that the frontier saved is not measurable on these trees. +Screens that recover to private AX are not affected. +The bound that remains is the raw node cap. +De-duplication drops a repeated node and re-parents its children onto that node's own parent. +Thus, identical rows collapse under one addressable owner instead of a subtree split across two nodes with the same identity. Scoped captures remain broad because depth is relative to the scope root selected in presentation. -Backend capability declarations describe how much acquisition work a regular depth request bounds. -They never determine whether a backend may answer it. -Every backend serves regular `--depth` requests because presentation cuts the acquired hierarchy. -Recursive tree and private AX enumerate their hierarchy before the cut (`presentation-cut`). -Flat query sweep has only its root and one presented level. -A cut beyond depth 1 leaves the sweep unchanged (`flat`). -`truncated` and `effectiveDepth` disclose completeness below an acquisition cap, as for unscoped capture. -A depth-capped regular capture is a subset of the same backend's unscoped capture. -Refusal previously left a plan pinned or deferred to private AX with only a synthetic sparse root. -The daemon then rejected it for a missing viewport (#2403). -For every backend, raw depth remains acquisition depth. +Backend capability declarations are part of the contract. +They describe how much acquisition work a regular depth request bounds, never whether the backend may answer it. +Every backend serves a regular `--depth` request because presentation applies the presented-depth cut to whatever hierarchy was acquired. +The recursive tree and private AX both enumerate their hierarchy and are cut afterwards (`presentation-cut`). +The flat query sweep has only its root and one presented level. +Thus, a cut past depth 1 returns the sweep unchanged (`flat`). +Completeness below an acquisition cap is disclosed in the same way as for an unscoped capture, through `truncated` and `effectiveDepth`, because a depth-capped regular capture is a subset of the unscoped capture from the same backend. +A refusal of the request instead produced no answer at all. +A plan pinned or deferred to private AX fell through to the synthetic sparse root, which the daemon then rejected as a missing viewport (#2403). +Raw depth remains acquisition depth for every backend. Acquisition-side limits remain explicit: -- Raw private-AX captures disclose the bridge-side node cap. -- Flat query sweep drops frameless elements because it has no hierarchy for geometryless semantics. -- Recursive tree has no raw-depth extension for deep XCTest trees. +- Raw private-AX captures still disclose their bridge-side node cap. +- The flat query sweep still drops frameless elements because it has no hierarchy to attach geometryless semantics to. +- The recursive tree still has no raw-depth extension for deep XCTest trees. -Presentation cannot repair these acquisition limits. +Presentation cannot repair any of those acquisition limits. -Before new iOS snapshot behavior is added, maintainers should decide which strategy owns it. -A change probably crosses strategy boundaries if it drops visible controls for a regular node budget or silently truncates raw snapshots. +When maintainers add new iOS snapshot behavior, they should first decide which strategy owns it. +A change probably crosses strategy boundaries if it tries to make regular snapshots fast by dropping visible controls behind a node budget. +A change also probably crosses strategy boundaries if it tries to make raw snapshots safe by silently truncating them. ## Amendment: in-place system surfaces (issue #2438) -Some UI appears outside the app's process in a system bundle. -For delegated OAuth/OIDC sign-in, `com.apple.SafariViewService` hosts `ASWebAuthenticationSession` and `SFSafariViewController`. -Two facts verified live on the iOS 26.2 Simulator determine capture behavior: +A system bundle presents some UI outside the app's process. +For example, `com.apple.SafariViewService` hosts `ASWebAuthenticationSession` and `SFSafariViewController` for delegated OAuth/OIDC sign-in. +Two facts, both verified live on the iOS 26.2 Simulator, decide how such UI is captured: -- **Activation destroys the surface.** `XCUIApplication.activate()` or `simctl launch` on the host cancels the authentication session and makes the view black. +- **The surface dies if activated.** `XCUIApplication.activate()` or `simctl launch` on the host cancels the authentication session and makes the view black. Thus, the host must be observed and operated **in place**, never activated. `open` refuses to launch a registered host. -- **The local host AX bridge cannot see the surface.** While the sheet appears, the app remains the AX `primaryApp`. - The bridge returns the occluded app tree as healthy. - Only the XCTest runner can read and operate the sheet through the host's bundle id. +- **The local host AX bridge cannot see the surface.** While the sheet is up, the app remains the AX `primaryApp`. + Thus, the bridge serves the (occluded) app tree as if it were healthy. + Only the XCTest runner, which addresses the host by bundle id, can read and operate the sheet. -Decision. A closed registry names these hosts in `contracts/fixtures/ios-system-surface-hosts.json`. -TypeScript and Swift registries mirror it under a parity test. +Decision. A closed registry names these hosts (`contracts/fixtures/ios-system-surface-hosts.json`). +The TypeScript and Swift registries mirror it under a parity test. When a registered host is genuinely presented, the runner serves and operates it in place. -It never adopts the host as the cached session target. -The session binding remains on the app. -After the surface disappears, the next command resolves to the app again. -On the Simulator, a low-cost device-scoped host-side probe detects a registered host process for the device. -It routes capture to the runner instead of the bridge. -Without a running host, the bridge fast path remains unchanged. +The runner never adopts the host as the cached session target. +The session binding stays on the app. +Thus, after the surface is gone, the next command resolves back to the app. +On the Simulator, a cheap, device-scoped host-side probe checks for a registered host process that runs for the device. +If such a host is running, the probe routes the capture to the runner instead of the bridge. +If no host is running, the bridge fast path is unchanged. A presented surface also takes precedence over an explicitly requested bundle id. -Before the runner resolves or activates `command.appBundleId`, it examines whether a host is presented. -Thus, even a command for a different app receives the sheet. -This is intentional because the sheet occludes the screen and leaves no readable content from the named app. -The capture discloses which surface it describes. -After the sheet disappears, no extra cost remains because the session binding never changed. - -Presence means `XCUIApplication.state == .runningForeground`, independently of tree content. -The live experiment showed that a stopped host could return a richer tree than a live host. -Thus, content heuristics cannot distinguish live and stopped hosts, but foreground state can. -A host can be foreground with a stale tree only after activation or relaunch. -The open guard and in-place policy both refuse those operations. -The never-activate guard therefore makes the foreground predicate valid. -The session never binds to the host. -Thus, delegated-auth flow cannot retain a stale host tree after teardown, the second bug in issue #2438. - -System-surface captures carry response-level `systemSurface` provenance and shared `iosSystemSurfaceDisclosure`, worded for each host kind. -This tells the agent that controls belong to a system sheet, such as web sign-in or Apple Pay, instead of the app. -Their lineage identifies the host instead of the app. +The runner checks for a presented host before it resolves or activates `command.appBundleId`. +Thus, a command that names a *different* app still receives the sheet. +This is intentional. +The sheet occludes the screen, so the named app has nothing readable under it, and the capture discloses which surface it describes. +After the sheet is gone, this costs nothing, because the session binding never moved. + +Presence is `XCUIApplication.state == .runningForeground`, not tree content. +The live experiment showed a torn-down host that still served a *richer* tree than a live host. +Thus, content heuristics cannot separate a live host from a dead host, but foreground state can. +A host can be foreground with a stale tree only if it was activated or relaunched. +The open guard and the in-place policy both refuse those operations. +Thus, this fix and the never-activate guard are one design: the guard makes the foreground predicate sound. +Because the session never binds to the host, issue #2438's second bug cannot occur for the delegated-auth flow. +That bug was a stale tree served confidently after teardown. + +Captures of a system surface carry a response-level `systemSurface` provenance and the shared `iosSystemSurfaceDisclosure`, worded per host kind. +Thus, the agent is told that the controls belong to a system sheet (web sign-in, Apple Pay) instead of the app. +These captures are also lineaged to the host instead of the app. Thus, their comparison identity differs from an app capture's identity by construction. -Ordinary key equality rejects cross-surface comparisons for every consumer. -Comparison sites need no separate surface checks. -Physical devices always use the runner, so in-place behavior applies without a route change. -Only the route probe is Simulator-specific. +Every consumer that asks "are these two captures the same presentation" refuses a cross-surface pair through ordinary key equality. +No comparison site carries a surface check of its own. +Physical devices always use the runner, so the in-place serve applies there without a route change. +The Simulator route probe is the only Simulator-specific piece. -The Apple Pay host, `com.apple.PassbookUIService`, belongs to the registry for text entry as well as snapshots. -Its billing, shipping, and contact forms contain text fields absent from the session app's tree. -A bare `type` addressed to the app process cannot see that keyboard. -The runner's first-responder route can type into those fields when it addresses the host in place. -This requires no changed text-entry branch. +The Apple Pay host (`com.apple.PassbookUIService`) joined the registry for text entry as much as for snapshots. +Its billing, shipping, and contact forms hold text fields that the session app's tree cannot resolve. +A bare `type` addressed to the app process never sees that keyboard. +When the runner addresses the host in place, its first-responder route can type into those fields. +No text-entry branch changed for it. ## Amendment: the coordinate space of a captured subtree (issue #2612) -Some system surfaces report geometry in the device's native portrait-up space while the app is rotated. -Their whole subtree uses that space. -Measurements used iPhone 17 Pro, iOS 26.2, and the system keyboard over a landscape app frame `(0,0,874,402)`. -`UIRemoteKeyboardWindow` reported `(0,0,402,874)`, with the app's side lengths exchanged. -The app window and `UITextEffectsWindow` both reported `(0,0,874,402)`. -In portrait, the spaces coincide and no rotation occurs. -Only the box identifies the subtree's reported space. -From the numbers alone, consumers cannot distinguish a landscape bottom keyboard from a strip down the left edge. -This caused refusals of uncovered app content and permitted a tap on a key. - -Decision. One published capture uses one space: the app's orientation space. -The producer resolves space while the platform frame still has its original context. -After capture, a merged tree has no axis for consumers to rotate against. -A window's own box declares its subtree's space. -That box is either the app's box or its quarter turn. -Non-window nodes inherit space from the window above them. -Published rectangles and their actionability verdicts use the same oriented frame. -Thus, an address and permission to tap it cannot disagree. - -When space cannot be identified, the capture publishes platform-reported geometry without an assumed rotation. -This applies when: +Some system surfaces keep their geometry in the device's native (portrait-up) space while the app is rotated. +Their *whole subtree* arrives turned with them. +A measurement used iPhone 17 Pro (iOS 26.2) with the system keyboard up over a landscape app frame `(0,0,874,402)`: + +- `UIRemoteKeyboardWindow` reports its own box as `(0,0,402,874)`, which is the app's box with its two side lengths swapped. +- The app's own window and `UITextEffectsWindow` both report `(0,0,874,402)`. + +In portrait, the two spaces coincide and nothing is turned. +Nothing but that box says which space a subtree reports in. +From the numbers alone, a consumer cannot tell a keyboard across the bottom of a landscape screen from a strip down its left edge. +Thus, the consumer refused app content far from the keyboard and let a tap land on a key. + +Decision. One published capture publishes one space: the app's orientation space. +The producer settles the space where the platform's frame is still the platform's frame. +The reason is that downstream of the capture, the question cannot be answered: a merged tree carries no axis that a reader could turn geometry against. +A window declares the space of its own subtree from its own box. +That box is either the app's box or that box turned through a quarter. +Anything that is not a window inherits the space of the window above it. +A published rect and the actionability verdict read from it come from the same oriented frame. +Thus, an address and the permission to tap it cannot disagree. + +In some cases, no space can be named: - No usable app frame exists. -- The platform does not report interface orientation. -- The app frame is too square to distinguish from its quarter turn. - -Consumers fail open on that geometry, as for other missing platform facts. -Detection and inverse normalization share one rotation table with synthesized dispatch. -This pure geometry belongs to `AgentDeviceSnapshotPresentation` in `SnapshotCoordinateSpace.swift`, outside the XCTest bundle. -Both languages replay `contracts/fixtures/window-coordinate-space.json` in ADR 0011's parity-table shape. -This ensures that capture applies the exact inverse of the forward dispatch rotation. - -Decision. A producer that cannot identify the app's interface orientation refuses the screen instead of publishing two spaces in one tree. -The Simulator AX bridge cannot identify that orientation. -The #2659 experiment, with verdict in that issue and details in #2667's diff, measured the only AX attribute for it. -`XC_kAXXCAttributeApplicationOrientation` (id 1503) resolves but reads 0 through the guest snapshot channel. -Through XCTest's reader, it returns `kAXErrorServerNotFound`. -The only low-cost service read gives device orientation, which differs from app orientation in a rotation-locked app. - -The decoder therefore counts window roots that report the app's box quarter-turned. -It fails that capture with `window-coordinate-space-unresolved`, kind `unsupported`. -This uses the same refusal shape as `remote-content-boundary`. -The route then serves the runner, which reads orientation. -A mixed-space tree cannot provide reliable partial geometry. -Each published rectangle is an address. -If some rectangles use a rotated axis, consumers cannot compare distances between arbitrary rectangles. -Every downstream consumer would need a rule to distrust selected windows. -Capture has the evidence to apply that rule itself. +- The platform did not report an interface orientation. +- The app frame is square enough to be indistinguishable from its own quarter turn. + +In these cases, the capture publishes what the platform reported instead of guessing a rotation. +Consumers fail open on that geometry, as they do on any missing platform fact. +Detection and the way back share one rotation table with synthesized dispatch. +That table is pure geometry that lives in the `AgentDeviceSnapshotPresentation` package (`SnapshotCoordinateSpace.swift`), not in the XCTest bundle. +Both languages replay it against `contracts/fixtures/window-coordinate-space.json` in the ADR 0011 parity-table shape. +This replay stops capture from turning back with anything other than the exact inverse of what dispatch turns forward. + +Decision. A producer that cannot name the app's interface orientation refuses the screen instead of publishing two spaces in one tree. +The Simulator AX bridge cannot name that orientation. +The #2659 spike (verdict on that issue, write-up in the diff of #2667) measured these facts: + +- The one AX attribute for it, `XC_kAXXCAttributeApplicationOrientation` (id 1503), resolves but reads 0 through the guest's snapshot channel. +- Through XCTest's own reader, that attribute errors with `kAXErrorServerNotFound`. +- The only cheap service read is *device* orientation, which diverges from the app's orientation on a rotation-locked app. + +Thus, the bridge's decoder counts window roots that report the app's box quarter-turned. +It fails that capture (`window-coordinate-space-unresolved`, kind `unsupported`). +This is the same refusal shape as `remote-content-boundary` above. +The route then serves the runner, which reads the orientation. +A mixed tree is not half-usable. +A published rect is an address. +After some rects in a tree answer to a turned axis, no pair of rects in that tree can answer "how far apart are these" any more. +Thus, every downstream consumer would have to know which windows to distrust. +The capture could have applied that rule and did not. The refusal costs one runner round trip and is correct. -Decision. This refusal has capture scope, not app-generation scope. -The generation circuit applies to failures that provide evidence about the app. -A web-view screen identifies a hybrid app, so its screens retain the runner until relaunch. -A rotated surface provides evidence only about the current screen. -It can disappear after the next keystroke. -If it retired the generation, every later portrait capture of a healthy app would use the runner because of one landscape keyboard. -Issue #2491 rejected that cost for a bridge still in preparation. -The failure code declares its scope. -Call sites do not decide it independently. - -The runner's recursive-tree capture measures the keyboard band instead of reconstructing it. -The same run can query `app.keyboards` and receive geometry in the app's orientation space (#2660). -This resolves the keyboard question on the path most dependent on this ADR's geometry. -The guard's landscape refusals therefore no longer depend on whether the tree arrived rotated. - -Capture still resolves space in one pass over the flat acquired array. +Decision. That refusal is scoped to the capture, not to the app generation. +The generation circuit is for failures that are evidence about the app. +A web-view screen is a property of a hybrid app, so its screens keep the runner until the app relaunches. +A rotated surface is evidence about the screen in front of the reader and about nothing else. +It is up now and gone after the next keystroke. +If the refusal retired the generation, every later portrait capture of a healthy app would move to the runner because of one landscape keyboard. +That is the cost that #2491 settled for a bridge that was only still in preparation. +Which side of that line a failure falls on is a declared property of its code, not a judgment made at the call site. + +The rest of this section states what stays unnormalized, and why the consumer rule stays. +The runner's recursive-tree capture now answers the keyboard with a measured band instead of a rebuilt band. +The reason is that the same run that owns the tree can ask `app.keyboards` and gets an answer in the app's own orientation space (#2660). +This closes the keyboard question on the path that used to depend most on this ADR's geometry. +It is also why the guard's landscape refusals no longer depend on whether the tree arrived turned. + +Capture still settles the space in ONE pass over the flat acquired array. `SnapshotGeometrySpace.normalized` runs once in `captureWithBackend`, between acquisition and presentation (#2661). -It uses each node's `type`, `parentIndex`, and `rect`. -The runner's query-sweep tier has no window ancestry, so its tree declares no native space. -Its flat children attach to a synthetic application root with the app's box. -The tier does not read interface orientation. -Thus, the uniform rule returns that tree exactly as reported. -This is a structural consequence, not a tier-specific exception. - -Provider producers `appium-source` and `limrun-ios-tree` also do not see app windows. -Their captures publish no keyboard fact. -Consumers interpret absence as a producer that did not measure the keyboard. -These paths publish platform-reported geometry. -The tap-path keyboard guard is the last consumer able to refuse geometry it cannot place. -Thus, its width rule remains. -The rule detects unnormalized input, not a universal iOS property. -The producers described above do normalize geometry. -A band taller than it is wide indicates a producer that did not normalize it. +The pass keys on each node's `type`, `parentIndex`, and `rect`. +The runner's query-sweep tier still has no window ancestry, so nothing in its tree declares a native space. +Its flat children hang off a synthetic application root that reports the app's own box. +That root reports the box under an interface orientation that the tier does not read. +Thus, the pass returns that tree exactly as reported. +This is a structural consequence of the same rule applied uniformly. +It is not the per-tier omission that a hand-threaded space would have made it. + +The provider producers (`appium-source`, `limrun-ios-tree`) never see the app's windows either. +A capture from any of them publishes no keyboard fact. +A consumer reads that silence as "this producer did not measure". +Those paths publish what the platform reported. +Thus, the last reader that can still refuse geometry that it cannot place is the tap-path keyboard guard, and its width rule therefore remains. +The rule detects un-normalized arrival, not a standing fact about iOS. +The producers that the earlier sections of this ADR describe do normalize. +A band taller than it is wide is what the output of a producer that did not normalize looks like. ## Amendment: modal containment in the presentation cut -The bridge reads one window and reports all attached containers. -XCTest queries return only the presentation available to the user. -React Navigation's card-plus-modal example provides measured evidence. +The bridge reads one window and reports every container attached to it. +XCTest's own queries answer only the presentation that the user can reach. +React Navigation's card-plus-modal example is the measured case. Three routes presented as sheets left three transition-view containers in one window. -One bridge capture published 567 nodes across three screens. -In the same app state, the runner published 76 nodes from only the top screen. -Maestro tests visibility geometrically. -Thus, `tapOn` resolved a covered screen's button, and the flow failed an assertion about a screen that never appeared. -A `presentation: "formSheet"` route's fields also matched intermittently (#2638). +One capture published 567 nodes across three screens. +The same app state under the runner published 76 nodes across the top screen alone. +Maestro's visibility test is geometric. +Thus, a `tapOn` resolved the covered screen's button, and the flow failed on an assertion about the screen that never arrived. +Also, the fields of a `presentation: "formSheet"` route matched intermittently (#2638). -Decision. The fold applies modal containment under all these conditions: +Decision. The fold applies modal containment when all of these conditions are true: -- A container's last transition view has UIKit's dimming view as a direct child. +- The last transition view under a container carries UIKit's dimming view as its direct child. - The producer reports that the dimming view takes touches (`userInteractionEnabled: true`). -- The dimmed area spans the frame of an earlier transition view. - -It removes those earlier transition views and their subtrees from regular and interactive projections. -Each condition is a producer fact, not an assumption about order. -UIKit's dimming view declares what it dims. -Its interaction state states whether a touch reaches content behind it. -Its frame supplies the rectangle that must contain the covered container. -These cases retain existing behavior: - -- No dimming view exists. -- The producer did not read the dimming view. -- The dimming view passes touches through. -- The earlier sibling is not a transition view. - -Containment applies only where the producer declares it. -The rule fails closed instead of guessing which siblings are covered. - -A sheet at an undimmed detent explains the interaction-state condition. -The relevant controls are `largestUndimmedDetentIdentifier` and react-native-screens `sheetLargestUndimmedDetentIndex`. -UIKit retains the dimming view with its window-sized frame, while the presenting screen remains reachable. -In react-navigation's form-sheet example, `Height Steps` on the presenting screen navigates while `Custom Dimming` rests at its smallest detent. -The bridge reports `userInteractionEnabled: false` for that dimming view. -It reports `true` for the dimmed form sheet and card modal. -No previously read attribute distinguishes them. -`XC_kAXXCAttributeIsVisible` also cannot distinguish them. -It reads false for both the undimmed sheet's dimming view and the card modal's dimming view. - -The bridge reads interaction state only for dimming views, with one subsequent read per view. -A read on every node added about half the capture time on a 492-node tree. -The targeted read did not change capture time. -Presentation counts every removed source in `stats.modalContainedNodeCount`. -Per ADR 0026, this is internal evidence, never wire vocabulary. -Without it, the same screen before and after animation can change hundreds of comparable lines without attribution. - -Semantic presentation rules (`IOS_PRESENTATION_RULES`) apply only to the interactive projection. -A suppression rule there could not provide the same behavior for regular `snapshot`. -The regular eligibility table has parity with its Swift equivalent. -An extension there would change a cross-language contract outside this rule's scope. -The occlusion pass marks covered nodes but retains them in publication. -Visibility filters read geometry, not that mark. -Thus, published-but-marked nodes retain the 567-versus-76 divergence. -A cut during acquisition would affect `--raw`, which must preserve the platform-reported tree. -Raw therefore retains covered screens. -These constraints place modal containment in the presentation cut. - -Producers without UIKit class names or dimming-view interaction state never trigger the cut. -This follows from their output facts, not backend exceptions in the fold. -The runner already omits modal-contained content from its queries. -`appium-source` and `limrun-ios-tree` report element types without classes. -The macOS desktop surface arrives already presented. -A scope that names a modal-contained screen publishes an empty projection. -This is the same healthy empty result as another unmatched scope because the named screen is not presented. + +Then the fold cuts the earlier transition views whose frame that dimmed area spans. +It cuts them, with their subtrees, from the regular and interactive projections. +Every part of the claim is a producer fact, not a guess from order: + +- The dimming view is UIKit's own declaration that it dims what sits behind it. +- Its interaction state says whether a touch there reaches what sits behind it. +- Its frame is the producer's rectangle, which is what a covered container must be inside. + +These cases keep today's behavior: + +- A presentation with no dimming view. +- A dimming view that the producer did not read. +- A dimming view that the producer reports as passing touches through. +- Any container whose earlier sibling is not a transition view. + +Containment is asserted only where the producer states it. +Thus, the rule fails closed instead of guessing which siblings are shadows. + +A sheet that rests at an undimmed detent is why the interaction state is part of the claim. +UIKit `largestUndimmedDetentIdentifier` and react-native-screens `sheetLargestUndimmedDetentIndex` configure such a detent. +UIKit keeps the dimming view in the tree at the same window-sized frame, and the presenting screen stays reachable. +In react-navigation's form-sheet example, a press on the presenting screen's `Height Steps` button navigates while the `Custom Dimming` sheet rests at its smallest detent. +The bridge reports that dimming view as `userInteractionEnabled: false`. +It reports `true` for the dimmed form sheet and for the card modal. +No attribute that the bridge already read told them apart. +`XC_kAXXCAttributeIsVisible` cannot tell them apart either. +It reads false for the undimmed sheet's dimming view and for the card modal's dimming view. + +The bridge reads the attribute for dimming views alone, in one follow-up read per view. +A request for it on every node increased the capture time by about half on a 492-node tree. +The targeted read did not change the capture time. +Every source that the cut removes is counted in the presentation's `stats.modalContainedNodeCount`. +Per ADR 0026, that count is internal evidence, never wire vocabulary. +The count exists because otherwise the same screen on either side of an animation moves hundreds of comparable lines with nothing to attribute them to. + +The neighboring seams do not fit this rule: + +- The semantic presentation rules (`IOS_PRESENTATION_RULES`) run only for the interactive projection. Thus, a suppression rule there could not deliver the same claim to a regular `snapshot`. +- The regular eligibility table is pinned against its Swift twin. Thus, a wider table would move a cross-language contract that this rule does not touch. +- The occlusion pass *marks* a node covered and keeps it published. Visibility filtering reads geometry, not the mark. Thus, a published-but-marked node is exactly the 567-versus-76 divergence that this rule fixes. +- A cut at acquisition would leak the decision into `--raw`, which owes the reader the tree that the platform reported. + +Raw therefore still carries the covered screens. + +Producers that report no UIKit class names or no dimming-view interaction state never trigger the cut. +That is a fact about their output, not a backend exception in the fold: + +- The runner already omits modal-contained content from its own queries. +- `appium-source` and `limrun-ios-tree` report element types and no classes. +- The macOS desktop surface arrives already presented. + +A scope that names a modal-contained screen now publishes an empty projection. +That is the same healthy empty answer that any other unmatched scope gives, because the screen is not what is presented. From ad6e3365ecff020a984a01a20ad350b5823368f1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 35/55] docs(adr): restore the frozen meaning in ADR 0005 Restore the ADR 0005 preflight-reduction gate as a present-tense rule and link the allowlisted uptime skip to it. Scope the RUNNER_BUSY glance rules so connect-failure replay is not banned. Restore dropped causal links, "not only from", "bookkeeping", and the base occupancy and close-and-pooling scopes. --- .../0005-ios-runner-interaction-lifecycle.md | 237 +++++++++--------- 1 file changed, 122 insertions(+), 115 deletions(-) diff --git a/docs/adr/0005-ios-runner-interaction-lifecycle.md b/docs/adr/0005-ios-runner-interaction-lifecycle.md index c23efc840e..ba6c213e20 100644 --- a/docs/adr/0005-ios-runner-interaction-lifecycle.md +++ b/docs/adr/0005-ios-runner-interaction-lifecycle.md @@ -4,49 +4,49 @@ Accepted -Completed process history is in git history. - ## Rules at a glance -- For iOS/macOS selector taps with a frame, use coordinate-first resolved element activation. tvOS remains focus/remote-driven. -- Before commands to ready sessions, probe `uptime`, except for the stated startup, readiness, and recent-mutation cases. -- Keep recent-mutation health only on the runner session and only for the stated app, commands, and 5-second window. +- For iOS/macOS selector taps on an element with a frame, use coordinate-first resolved element activation. tvOS remains focus/remote-driven. +- Before a command to a ready session, send an `uptime` preflight, except for the stated startup, readiness, and recent-mutation cases. +- Keep recent-mutation health only on the `RunnerSession` and only for the stated app, commands, response checks, and 5-second window. +- Never count snapshots or other read-only responses as health signals. +- Future optimization work should reduce the `uptime` preflights only after runner status survives command-induced XCTest teardown and can prove that the session still serves new requests. - Invalidate dead cached runner processes without graceful `shutdown`. -- After `RUNNER_BUSY`, resend only read-only commands under their busy-specific budget. Never replay mutations. -- Before a busy runner can enter retention, `close` stops it and awaits lease release. +- After `RUNNER_BUSY`, resend only read-only commands, under their busy-specific budget. +- After `RUNNER_BUSY` on a mutating command, return the refusal to the caller without a resend. +- If `close` would retain a runner whose main-thread occupancy is set, stop that runner first and await the stop. - After a root AX failure, clear the cached app target. Keep raw/strict errors distinct from interactive recovery payloads. ## Context The iOS runner is a long-lived XCTest process with an HTTP command loop. -A command can appear complete at the daemon boundary while XCTest already stops the runner. +A command can appear to complete at the daemon boundary while XCTest is already tearing down the test runner. -This occurred in the React Navigation playground after selector taps that caused navigation, such as `Navigate to Details` and `Back to home`. +This failure was reproduced in the React Navigation playground with selector taps that cause navigation, such as `Navigate to Details` and `Back to home`. The runner resolved the button and synthesized the tap. The app navigated, and XCTest then tried to resolve the original `XCUIElement` again. Because the element had disappeared, xcodebuild recorded `Failed to get matching snapshot` and ended the test with `** TEST EXECUTE FAILED **`. The daemon had already received a successful tap response. -Thus, the next read-only command used a stale cached runner. +Thus, the next read-only command inherited a stale cached runner. These earlier assumptions were incorrect: -- A recent successful runner response proves continued runner health. -- After selector resolution, `XCUIElement.tap()` is the safest selector-tap primitive. -- A cached `XCUIApplication` target remains safe after XCTest cannot serialize the app's accessibility tree. +- A recent successful runner response proves that the runner is still healthy. +- After a selector has resolved, `XCUIElement.tap()` is the safest selector-tap primitive. +- A cached `XCUIApplication` target remains safe after XCTest reports that it cannot serialize the app's accessibility tree. ## Decision Coordinate-first resolved element activation is the iOS/macOS selector-tap model. -The runner uses selectors or text queries to find the semantic `XCUIElement`. -If the element has a frame, activation taps its resolved center point instead of calling `XCUIElement.tap()`. -tvOS remains focus/remote-driven because it does not support normal coordinate input. +The runner still uses selectors or text queries to find the semantic `XCUIElement`. +If the element has a frame, activation taps the resolved center point instead of calling `XCUIElement.tap()`. +tvOS remains focus/remote-driven because tvOS does not support normal coordinate input. Before command send, ready runner sessions receive a short `uptime` preflight. -Read-only startup commands skip this preflight. -The first successful command proves readiness for a newly launched runner. -Readiness probe commands also skip preflight to avoid recursion. +Read-only startup commands still skip that preflight because, for a newly launched runner, the first successful command is the readiness proof. +Readiness probe commands skip preflight to avoid recursion. -The daemon may also skip ready-session `uptime` for an explicit allowlist of mutating interactions: +The daemon may also skip the ready-session `uptime` preflight for an explicit allowlist of mutating interactions: - `tap`. - `longPress`. @@ -55,114 +55,118 @@ The daemon may also skip ready-session `uptime` for an explicit allowlist of mut - `scroll`. - `sequence`. -This requires a healthy mutating response from the same session for the same `appBundleId` within 5 seconds. -The response must parse successfully and carry no `runnerFatal`. -Only the `RunnerSession` object stores this recency as `lastHealthyMutation`. -Each invalidation or restart deletes it. +The skip requires that the same session produced a healthy mutating response for the same `appBundleId` within 5 seconds. +A healthy mutating response is parsed ok and does not carry `runnerFatal`. +This recency lives only on the `RunnerSession` object as `lastHealthyMutation`. +Thus, it is discarded with every invalidation or restart. The record is written only after the `runnerFatal` check. -Sparse AX-fallback snapshots and `runnerFatal` payloads therefore never renew it. -Snapshots and other read-only responses never count as health signals. - -This narrow skip is permitted because coordinate-first activation removes the command-induced teardown trigger. -The lifecycle status journal and status-before-invalidate recovery preserve status across teardown. -They resolve ambiguous post-send failures before invalidation. -After a skipped preflight, a transport failure clears recency and carries skip context. -Connection-shaped failures, such as refused, reset, or hung up, use status recovery instead of blind replay. -Timeout-shaped failures propagate with skip context. -This is the same classification used by preflighted sends. - -`uptime` directly probes the runner listener. -The listener answers before command journaling, the serial execution queue, app activation, and main-thread XCTest dispatch. -It should measure only whether the runner is alive and accepts new HTTP requests. - -Invalidate dead cached runner processes without graceful `shutdown`. -A stopped process cannot answer a shutdown request. -Graceful cleanup would only add stale-listener delay. - -Every successful runner response carries live main-thread occupancy as `runnerMainThreadBusy`. -An execution-watchdog timeout carries typed code `MAIN_THREAD_TIMEOUT`. -Thus, the stalled command itself informs the daemon of occupancy before a later `RUNNER_BUSY` refusal. -The daemon mirrors occupancy on `RunnerSession`: - -- Set it on `RUNNER_BUSY` or `MAIN_THREAD_TIMEOUT`. -- Clear it on any other served runner reply, which reached the main thread and therefore drained. -- Preserve it after a transport failure or unstamped recovered response. - -A healthy `ok` never proves drain. -A private-AX snapshot can succeed while an abandoned tree crawl continues on the XCTest main thread (#2552). - -The daemon's per-command read-only trait determines behavior after `RUNNER_BUSY`. -No runner-side command classification controls it. -`RUNNER_COMMAND_TRAITS` in `packages/platform-apple/src/runner/runner-command-traits.ts` owns this trait. -`runAppleRunnerCommand` reads it through `isReadOnlyRunnerCommand` when it selects the resend loop. -`runner-client-busy-resend.test.ts` asserts these distinct behaviors: - -- A read-only command waits for drain under a busy-specific budget sized to outlast it. +Thus, sparse AX-fallback snapshots and `runnerFatal` payloads never refresh it. +Snapshots and other read-only responses never count as a health signal. + +This narrow skip is permitted because it meets the preflight-reduction precondition in the Consequences section: + +- Coordinate-first activation removed the command-induced teardown trigger. +- The lifecycle status journal plus the status-before-invalidate recovery is the teardown-surviving status surface. + That surface resolves any ambiguous post-send failure before invalidation. + +After a skip, a transport failure clears the recency record and is marked with the skip context. +Connection-shaped failures (refused, reset, hung up) run status recovery instead of a blind replay. +Timeout-shaped failures propagate with the skip context. +This is the same classification that preflighted sends use. + +`uptime` is a direct runner listener probe. +The listener answers it before command journaling, the serial command execution queue, app activation, and main-thread XCTest dispatch. +The `uptime` probe should measure only whether the runner is alive and accepts new HTTP requests. + +Dead cached runner processes are invalidated without graceful `shutdown`. +A process that already stopped cannot answer the shutdown request. +Thus, graceful cleanup only adds stale-listener delay. + +The runner stamps its live main-thread occupancy (`runnerMainThreadBusy`) onto every successful response. +The runner tags the execution-watchdog timeout with the typed `MAIN_THREAD_TIMEOUT` code. +Thus, the daemon learns of main-thread occupancy from the stalling command itself, not only from a later `RUNNER_BUSY` refusal. +The daemon mirrors that occupancy on the `RunnerSession`: + +- `RUNNER_BUSY` or `MAIN_THREAD_TIMEOUT` sets it. +- Any other served runner reply clears it, because that reply reached the main thread and therefore drained. +- A transport failure or an unstamped recovered response leaves it intact. + +The daemon never reads a healthy `ok` as proof of drain. +The reason is that a private-AX snapshot can be served while an abandoned tree crawl still continues on the XCTest main thread (#2552). + +The cost of a `RUNNER_BUSY` refusal to the caller depends on the daemon's own per-command read-only trait. +It does not depend on any runner-side classification of the command. +The trait is `RUNNER_COMMAND_TRAITS` in `packages/platform-apple/src/runner/runner-command-traits.ts`. +`runAppleRunnerCommand` reads it through `isReadOnlyRunnerCommand` when it decides whether to wrap the send in a resend loop. +This asymmetry is shipped, and `runner-client-busy-resend.test.ts` asserts it: + +- A read-only command treats the refusal as *not yet*. + It waits for the drain on the busy-specific budget sized to outlast the drain. It resends until the runner answers or the window ends. -- A mutating command makes one attempt. - The refusal reaches the caller without a wrapper or replay. - -Neither path performs a status probe because the structured reply already states whether the command ran. -`runner-lifecycle.ts` implements this bypass. -The same trait controls the startup-preflight skip above. -It is therefore the single owner for classification changes. - -If `close` would retain an occupied runner for reuse, it first stops that runner. -It awaits the stop so the lease is released before the next request. -A runner with watchdog-abandoned work refuses every command until drain or escalation to `RUNNER_WEDGED`. -This fact governs close and retention only. -The read-only resend loop is one live caller that waits under its own budget. -`close` does not inherit that wait for the pool. -Returning the stalled runner to the pool gives it to the next `open` without recovery. -Only process termination can abort uncancellable XCTest work. -This applies the `RUNNER_WEDGED` restart from #1105 at close instead of after the wedge threshold. +- A mutating command treats the same refusal as *not mine to send again*. + It makes one attempt, and the refusal reaches the caller unwrapped, with nothing replayed. + +Neither path sends a status probe, because a structured reply already answered "did my command run?". +This bypass is in `runner-lifecycle.ts`. +The same trait already decides the startup-preflight skip above. +Thus, it is the single place for a change to that classification. + +If `close` would retain a runner for reuse and that occupancy is set, `close` first stops the runner. +`close` awaits the stop, so the lease is released before the next request. +A runner that still finishes watchdog-abandoned work refuses every command sent to it until it drains or escalates to `RUNNER_WEDGED`. +This ADR states that fact only for the close-and-pooling decision. +The read-only resend above is one live caller that waits out the same refusal on its own budget. +`close` does not inherit that wait on behalf of the pool. +If the pool takes the stalled runner back, it gives that runner to the next `open`, and `close` recovers nothing. +Killing the process is the only way to abort uncancellable XCTest work. +This is the `RUNNER_WEDGED` restart from #1105, applied at the close boundary instead of after the wedge threshold elapses. If XCTest reports a root accessibility snapshot failure such as `kAXErrorIllegalArgument`, the runner treats the cached app target as suspect. -Interactive snapshots fail closed with a truncated root-only payload. -They do not issue more flat fallback queries against the broken tree. +Interactive snapshots fail closed to a truncated root-only payload. +They do not issue more flat fallback queries against the same broken tree. The runner clears the cached `XCUIApplication` handle. -The next command reacquires the target through normal activation. +Thus, the next command reacquires the target through the normal activation path. An external iOS-family Simulator relaunch or physical iOS relaunch also invalidates process-bound target state. -After app process replacement, the daemon sends a lifecycle reset to the retained runner. -The next command then reacquires `XCUIApplication`. +After it replaces the app process, the daemon sends a lifecycle reset to the retained runner. +Thus, the next command reacquires `XCUIApplication`. The reset also clears process-bound snapshot penalty and private-AX depth state. -If the reset cannot be confirmed, the daemon discards the runner session. -If relaunch fails before reset, it also discards the session. -Other physical Apple OS leaves retain existing runner-restart behavior until separate evidence supports a change. +If that reset cannot be confirmed, or if relaunch fails before reset, the daemon discards the runner session. +Other physical Apple OS leaves retain their existing runner-restart behavior until separate evidence exists. -The snapshot surface intentionally has two AX-failure forms. -Interactive fast snapshots return truncated success with `runnerFatal`. -Agents can see AX unavailability and recover through a plain screenshot and coordinate navigation. -Raw or strict snapshot paths retain errors because their callers request a faithful tree instead of lossy recovery. +The snapshot surface intentionally has two AX-failure shapes. +Interactive fast snapshots return a truncated success payload with `runnerFatal`. +Thus, agents can still see that AX state is unavailable and recover with a plain screenshot plus coordinate navigation. +Raw or strict snapshot paths continue to return an error, because those callers requested a faithful tree, not a lossy recovery payload. ## Consequences -Selector taps that cause navigation no longer depend on XCTest's post-tap element resolution. -If navigation removes the target, the tap remains a normal successful interaction. -The runner should remain alive. +The success of a selector tap that causes navigation no longer depends on XCTest's post-tap element bookkeeping. +If the target disappears because navigation happened, the tap remains a normal successful interaction. +The runner should stay alive. -If xcodebuild exits for another reason, process/liveness checks detect the stale runner on the next command. -They avoid the old 15-second graceful-shutdown wait. -Remaining latency comes from fresh xcodebuild runner startup instead of stale transport delay. +If xcodebuild still exits for another reason, the next command detects the stale runner through process/liveness checks. +That command also avoids the old 15-second graceful-shutdown wait. +The remaining latency is fresh xcodebuild runner startup, not a stale transport stall. -A `close` then `open` during abandoned main-thread work starts a fresh runner instead of reusing the stalled runner. -The wedge therefore does not persist across close/open. -When retention is requested, ready runners without unanswered exchanges can remain active for reuse. +A `close` then `open` on a runner that still drains abandoned main-thread work now starts a fresh runner. +It does not inherit the stalled runner. +Thus, the wedge no longer survives the close/open cycle. +When retention is requested, runners that are ready and have no unanswered exchanges can remain warm. iOS-family simulators also require an observed live listener. Close retires unfinished startup and its recovery authority. The retained generation owns its idle timer and listener observation. Successful reuse ends retention. Listener loss or idle expiry disposes that generation. -Graceful daemon shutdown stops idle retained generations before exit. +Graceful daemon shutdown stops idle retained generations before it exits. Active ready generations retain the existing handoff policy. -The daemon does not treat generic recent success as runner health. -A proven healthy mutating response for the same app provides end-to-end liveness evidence from HTTP listener to app target. -It counts only after the `runnerFatal` check and only for allowlisted interactions. -A repeated series of allowlisted interactions can therefore skip per-command `uptime`. -Each healthy mutation establishes permission for the next skip. -Unconditional `uptime` remains the default for non-allowlisted commands and after: +The daemon no longer models a generic "recent success" cache as a runner-health signal. +A proven healthy mutating response for the same app is now a real end-to-end liveness proof, from the HTTP listener through to the app target. +That response is recorded only after the `runnerFatal` check and only for allowlisted interactions. +Thus, a repeated series of allowlisted interactions skips the per-command `uptime` request. +Each skip is still earned again from another healthy mutation. +The earlier unconditional `uptime` before every ready-session command remains the default for non-allowlisted commands and after any of these: - Invalidation. - A stale record. @@ -170,14 +174,17 @@ Unconditional `uptime` remains the default for non-allowlisted commands and afte - An absent record. XCTest may still be unable to examine deeply an app with a broken accessibility tree. -However, one failed snapshot no longer causes continued use of a suspect target or additional failures from every interactive element query. +However, one failed snapshot no longer causes the runner to keep using a suspect cached app target. +It also no longer causes the runner to amplify the failure with a walk of every interactive element query. -After a tap, `XCTEST_RECORDED_FAILURE` is an ambiguous outcome at the daemon boundary. +An `XCTEST_RECORDED_FAILURE` returned after a tap is treated as an ambiguous outcome at the daemon boundary. If a usable retained snapshot exists, the daemon takes one same-presentation post-action capture. -A changed accessibility digest returns success with an explicit warning. -This prevents blind repetition of a tap that may already have navigated. -Unchanged, sparse, mismatched, or unavailable evidence retains the original failure. +A changed accessibility digest is reported as success with an explicit warning. +Thus, agents do not blindly repeat a tap that may already have navigated. +Unchanged, sparse, mismatched, or unavailable evidence remains the original failure. + +iOS-family Simulator and physical iOS relaunch keep the healthy XCTest process warm without carrying an app target across process identity. +The reset adds one local runner request instead of the cost of a runner restart. +Before the replacement target is reacquired, the reset clears the old process's hostile-screen capture penalty. -iOS-family Simulator and physical iOS relaunch retain a healthy XCTest process without retaining an app target across process identity. -The reset adds one local runner request instead of a runner restart. -Before replacement target acquisition, it clears the old process's hostile-screen capture penalty. +Future optimization work should reduce these `uptime` preflights only after the runner exposes status that survives command-induced XCTest teardown and can prove that the session still serves new requests. From ffa26604dc8cedb932ba5a045bf7be2bdf818b5c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 36/55] docs(adr): restore the frozen meaning in ADR 0006 Restore "may" and the proxied-upstream condition in the ADR 0006 cache glance rule, and add the must-honor-conditional-headers rule to the glance. State the one remaining wire-compat gap. Restore "fail before it sends", "is treated as", "instead of failing clearly", and the "over bumping the whole protocol" preference. --- docs/adr/0006-daemon-rpc-protocol-version.md | 89 ++++++++++---------- 1 file changed, 45 insertions(+), 44 deletions(-) diff --git a/docs/adr/0006-daemon-rpc-protocol-version.md b/docs/adr/0006-daemon-rpc-protocol-version.md index 8cca391e8c..432ae7a8f3 100644 --- a/docs/adr/0006-daemon-rpc-protocol-version.md +++ b/docs/adr/0006-daemon-rpc-protocol-version.md @@ -7,97 +7,98 @@ Accepted ## Rules at a glance - Before remote JSON-RPC requests, compare the daemon's `rpcProtocolVersion` from `/health`. -- Package `version` is diagnostic only and must not control compatibility. -- Permit missing `rpcProtocolVersion` unless a later security or protocol decision explicitly retires legacy compatibility. -- Cache probes only with complete endpoint and upstream instance IDs. Send conditional headers on every cached RPC. -- After a stale-instance refusal, probe again. Retry once only if the new peer's protocol is compatible. -- Change `rpcProtocolVersion` only for unsafe compatibility changes to existing HTTP RPC contracts. -- Keep additive changes at the existing protocol version. +- Package `version` is diagnostic only and must not be used as a compatibility gate. +- Allow a missing `rpcProtocolVersion` unless a later security or protocol decision explicitly retires legacy compatibility. +- A persistent client may cache a successful probe if `/health` advertises an instance ID for the daemon endpoint and, for a proxy, for its upstream daemon. +- Send those instance IDs as conditional headers on every cached RPC. Peers without complete instance IDs retain per-command probes. +- After a stale-instance refusal, clear the cache and probe `/health` again. Retry the refused RPC once, and only if the new peer's protocol is compatible. +- A peer that advertises an instance ID must continue to honor the conditional RPC headers in future versions. +- Change `rpcProtocolVersion` only when older and newer peers cannot safely communicate over the HTTP RPC boundary for existing commands. +- Do not change `rpcProtocolVersion` for additive changes. - Treat an absent `capabilities` entry in `daemon.json` as "cannot do this". Never infer a capability from the package version. - A client never signals a daemon that did not accept a quiesce, except an unreachable daemon it retires deliberately. -- Force-retire a daemon at the drain deadline only when it fails its liveness probes and its command line proves it is a daemon process. +- After `draining`, force-retire a daemon at the deadline only when it is still running, fails its liveness probes, and its command line proves it is a daemon process. ## Context -`agent-device` can run a client on one machine and connect to a daemon on another through HTTP JSON-RPC. +`agent-device` can run a client on one machine and reach a daemon on another machine through the HTTP JSON-RPC transport. This includes `agent-device proxy` in front of a local macOS daemon. Client and remote host package versions can differ. Many version differences are safe because the client sends a stable command request envelope and the daemon executes only the requested command. Package semver is too strict for this boundary. A 0.15 client and 0.17 remote daemon can often interoperate. -However, the client needs a low-cost compatibility check before command RPC when the transport protocol is incompatible. +However, the client needs a low-cost way to fail before it sends command RPC if the transport protocol itself is no longer compatible. ## Decision -The daemon HTTP `/health` payload advertises integer `rpcProtocolVersion` beside service and package version metadata. -Before JSON-RPC requests, remote clients compare this value. +The daemon HTTP `/health` payload advertises a `rpcProtocolVersion` integer beside service and package version metadata. +Remote clients compare this value before they send JSON-RPC requests. -Package `version` is diagnostic only and must not control compatibility. -Missing `rpcProtocolVersion` identifies a legacy remote daemon. -It is permitted unless a later security or protocol decision explicitly retires legacy compatibility. +Package `version` is diagnostic only and must not be used as a compatibility gate. +A missing `rpcProtocolVersion` is treated as a legacy remote daemon. +A legacy remote daemon is allowed unless a later security or protocol decision explicitly retires legacy compatibility. -A persistent client may cache a successful probe if `/health` advertises instance IDs for the daemon endpoint and any proxied upstream daemon. -On every cached RPC, it sends those IDs as conditional headers. +A persistent client may cache a successful probe if `/health` advertises an instance ID for both the daemon endpoint and, if proxied, its upstream daemon. +The client sends those IDs as conditional headers on every cached RPC. Before dispatch, a daemon refuses a stale instance with a typed HTTP 409 response. Before it forwards a request, a proxy refuses its own stale instance. -The proxy passes the expected upstream instance to the daemon, which performs the same pre-dispatch check. -After a mismatch, the client clears the cache and probes `/health` again. -Only if the new peer's protocol is compatible does it retry the refused RPC once. +The proxy passes the expected upstream instance to the daemon, which makes the same pre-dispatch check. +On a mismatch, the client clears the cache and probes `/health` again. +The client then retries the refused RPC once, and only if the new peer's protocol is compatible. Peers without complete instance IDs retain per-command probes. -A peer that advertises an instance ID must continue to honor conditional RPC headers in future versions. -Older clients may retain a cached identity across a restart. -Before retrying a mutating command, they must receive a pre-dispatch refusal. +A peer that advertises an instance ID must continue to honor the conditional RPC headers in future versions. +The reason is that older clients may hold a cached identity across a restart. +Such clients must receive a pre-dispatch refusal before they retry a mutating command. -`rpcProtocolVersion` changes only when an older client and newer daemon, or newer client and older daemon, cannot safely communicate for existing commands. -This applies to the HTTP RPC boundary. +`rpcProtocolVersion` changes only when an older client and newer daemon, or a newer client and older daemon, cannot safely communicate over the HTTP RPC boundary for existing commands. Increment it for incompatible changes to: - HTTP route requirements for `/health`, `/rpc`, `/upload`, or `/artifacts/*`. - Authentication semantics required to authorize RPC, upload, or artifact requests. - JSON-RPC envelope shape, method names, request id handling, or command request projection. -- Response, error, artifact, upload, or progress-stream framing parsed by existing clients. -- Existing command request or response contracts that the older side would misinterpret instead of clearly rejecting. +- Response, error, artifact, upload, or progress-stream framing that existing clients parse. +- Existing command request or response contracts, if the old side would misinterpret the payload instead of failing clearly. Do not increment it for additive changes: - New commands. - New optional request fields or flags that older daemons can ignore or reject with a normal command error. - New optional response fields that older clients can ignore. -- Package version changes, refactors, or daemon/proxy changes limited to implementation. +- Package version changes, refactors, or implementation-only daemon/proxy changes. -Unless the shared transport contract also changes, prefer command-level metadata for finer command compatibility. -Feature or capability metadata can supply this information. +If a single command needs finer-grained compatibility in the future, prefer command-level feature or capability metadata to an increment of the whole RPC protocol. +This preference does not apply if the change also affects the shared transport contract. ## Alternatives Considered -- **Use package version for compatibility.** This is simple but rejects compatible version differences. - Frequent releases also make proxy use unreliable across versions. -- **Do not check compatibility.** This maximizes attempted compatibility. - However, failure occurs after the client sends an RPC payload that a remote daemon may parse incorrectly. -- **Negotiate full schemas.** This is more precise but requires excessive machinery for the current JSON-RPC boundary. +- **Gate by package version.** This is simple, but it rejects compatible version differences. + It also makes proxy use brittle across frequent releases. +- **Do not check compatibility.** This maximizes compatibility. + However, it fails later, after the client has sent an RPC payload that a remote daemon may parse incorrectly. +- **Negotiate full schemas.** This is more precise, but it is too much machinery for the current JSON-RPC boundary. ## Consequences Protocol-breaking changes must update: - `DAEMON_RPC_PROTOCOL_VERSION`. -- Tests for `/health` metadata. +- Tests that assert `/health` metadata. - At least one remote-client regression test that proves mismatched protocols fail before command RPC. -Checks enforce the version rules above (#1432). -`test/wire-compat/surface.ts` declares the wire surface, grouped by the decision's categories. -`test/wire-compat/ledger.json` records each declaration's hash and protocol version. -The unit lane fails if a declaration's shape changes without a corresponding ledger change. +Checks enforce the increment rules above, so nobody has to remember them (#1432). +`test/wire-compat/surface.ts` declares the wire surface, grouped by the bullets in this decision. +`test/wire-compat/ledger.json` records the hash of each declaration and the protocol version of that hash. +The unit lane fails if the shape of a declaration changes and the ledger does not change with it. `pnpm check:daemon-wire-compat` compares the ledger with the last released tag. -Changes since that release must have either a protocol increment or a `compatibleChanges` acknowledgment under the additive list above. -Some categories are only partly represented in digests, including `/health` and `/rpc` path literals inside `http-server.ts`. -For those categories, the manifest records the gap and its reason instead of implying coverage. +Drift since that release must carry either a protocol increment or a `compatibleChanges` acknowledgment that invokes the additive list above. +One bullet is only partly digestible: the `/health` and `/rpc` path literals inside `http-server.ts`. +For that gap, the manifest records the gap and its reason instead of implying coverage. See `test/wire-compat/README.md`. Legacy remote daemons without `rpcProtocolVersion` remain reachable. -This keeps the first proxy release compatible with older HTTP daemons. -However, an absent marker does not prove compatibility. +This keeps the first release of the proxy compatible with older HTTP daemons. +However, the absence of the marker is not proof of compatibility. ## Daemon capabilities and quiesce From d9f7634316b57af165fab78967b7ec245b2e58a4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 37/55] docs(adr): restore the frozen meaning in ADR 0007 Restore the caller-named window and the close-release case in the ADR 0007 glance rules, and fix the "It allocates" referent. Restore the full request-lock list, past-tense incident facts (#2946, orphaned allocations, expired admitted work), the dropped heartbeat-default history, and the base terms "treated as", "is refused with", "is used". --- docs/adr/0007-remote-device-leases.md | 370 +++++++++++++------------- 1 file changed, 187 insertions(+), 183 deletions(-) diff --git a/docs/adr/0007-remote-device-leases.md b/docs/adr/0007-remote-device-leases.md index 9a4100a027..96b0abf5f5 100644 --- a/docs/adr/0007-remote-device-leases.md +++ b/docs/adr/0007-remote-device-leases.md @@ -6,282 +6,286 @@ Accepted ## Rules at a glance -- Allocate device leases lazily after device, backend, and provider resolution. -- Keep lease mutation and request execution under the same daemon request lock. +- Allocate device leases lazily, when the device, backend, and provider are known. +- Run lease admission, heartbeat, stored session lease refresh, and request execution under the same daemon request lock. +- Do not mutate lease ownership before the lock. Scope resolution may happen before the lock. - Generated connection profiles must strip daemon and Metro bearer tokens. -- A session cannot outlive its lease. Teardown clears its device claim even if the device does not answer. -- Report request-ended leases once on the outermost response. Clients must not release them again. -- Wanted admitted work preserves its lease. Canceled work neither defers expiry nor renews it. -- Before admission, protect long client phases with lease heartbeats under the lease's existing window. -- Human-control holds fence mutations through registry-owned admission and drain behavior. -- Only host administrators allocate and release `macos-app` leases. Confine each admitted request to the leased app. +- A session cannot outlive its lease. Lease-end teardown clears the session's device claim even when the device no longer answers. +- Report each lease that a request ended once, at response level. A client never sends a release after such a request. +- Admitted work that is still wanted preserves its lease. If the client hung up, the request neither defers expiry nor renews the lease. +- Before admission, protect long client phases with lease heartbeats. Renew under the window that the caller named, or else under the lease's existing window. +- Before a pending human-control activation reports active, it fences new mutations and drains the mutations already admitted. +- Only host administrators allocate `macos-app` leases. Tenant `lease_release` is refused. +- Confine each request admitted under a `macos-app` lease to the leased app. ## Context Remote daemon users need a clear ownership boundary before commands reach a platform runner or helper. -Shared proxies and hosted providers need ownership to include the selected device and connection provider, in addition to tenant/run. +Shared proxies and hosted providers need ownership to include the selected device and connection provider, not only tenant/run. Runner and helper processes already have backend-specific mutual exclusion. -This guard protects platform tools, not remote client ownership. -Direct exposure of those errors makes device contention harder to resolve. +That guard protects platform tooling, not remote client ownership. +Thus, direct exposure of those errors makes device contention harder to recover from. ## Decision -A remote device lease is logical ownership of one selected device by one remote client for a connection provider. -Examples include `proxy`, cloud, and `limrun`. +A remote device lease is logical ownership of one selected device by one remote client for a connection provider such as `proxy`, cloud, or `limrun`. `connect` establishes the connection profile and client identity. Lease allocation is lazy. -It occurs after device, backend, and provider resolution. +It happens when a device, backend, and provider are known. -A runner/process lease guards a backend helper. -It does not define user/client ownership. -It remains below daemon device leases and should not be weakened or replaced by them. +A runner/process lease is a backend helper guard. +It is not a user/client ownership boundary. +It stays below daemon device leases and should not be weakened or replaced by them. -`open` is the natural place to acquire a device lease because target resolution and session creation meet there. -After `open`, commands must refresh the lease. +`open` is the natural point to acquire a device lease because target resolution and session creation meet there. +Commands after `open` must refresh the lease. After five minutes without activity, the device should become available again. -These operations must use the same daemon request lock: +These operations must run under the same daemon request lock: - Lease admission. - Heartbeat. - Stored session lease refresh. - Request execution. -Scope resolution may occur before the lock. -Lease ownership mutation must not. +Scope resolution may happen before the lock. +Lease ownership mutation must not happen before the lock. Generated connection profiles are non-secret. -They may persist route and lease metadata, but must strip daemon and Metro bearer tokens. +They may persist routing and lease metadata, but must strip daemon and Metro bearer tokens. Tokens are supplied in memory for the current command or through environment/CLI token paths. -The proxy process is expected to remain active for self-service use. -Recovery from stale or expired device leases should not require a proxy restart. +The proxy process is expected to be long-lived and self-serve. +Recovery from a stale or expired device lease should not require a proxy restart. -A caller that allocates and releases a lease itself can allocate with `retainOnClose`. -Session `close` then retains that lease and its provider device. +A caller that owns the lifetime of a lease, and allocates and releases it itself, can allocate with `retainOnClose`. +Session `close` then leaves that lease and its provider device in place. Only `leases.release`, expiry, or daemon shutdown ends it. -The default remains unchanged, so CLI proxy sharing still frees devices on `close`. +The default is unchanged, so the CLI's proxy sharing still frees devices on `close`. The allocated lease reports `retainOnClose: true` only if the daemon honored it. -Only the lease's own client can enable it for a lease already held by the run. +Only the lease's own client can turn it on for a lease that the run already holds. An unexpired `retainOnClose` lease keeps an idle daemon alive. Other leases, including human-control holds, do not. -If the caller stops heartbeats, a retained lease expires after its `ttlMs`. -Idle reap then stops the daemon within one more idle window. +If the caller stops heartbeats for a retained lease, that lease expires after its `ttlMs`. +Idle reap then shuts the daemon down within one more idle window. -A session cannot outlive the lease under which it opened. -On release or expiry, the daemon tears the session down under its execution locks and clears its device claim. +A session cannot outlive the lease under which it was opened. +When the lease is released or expires, the daemon tears the session down under its execution locks and clears its device claim. This applies even when the device no longer answers. -Release can use tenant `lease_release` or host `DELETE /admin/leases/`. -The daemon examines expired leases every second. -Thus, expiry does not require a later request. -In-flight work and human-control holds still prevent lease expiry. -After daemon shutdown starts, no new lease-end teardown starts. -Shutdown waits for already-started teardown before it tears sessions down itself. - -When a request closes a session, the daemon ends its lease unless it was allocated with `retainOnClose`. -The response reports each lease ended by that request once, at response level: -`endedLeases: [{ leaseId, outcome: 'released' }]`. -Success reports it in `data`. -Failure reports it in `error.details`, including when close fails after release. +A release is a tenant `lease_release` or a host `DELETE /admin/leases/`. +The daemon sweeps for expired leases every second. +Thus, expiry does not wait for a later request. +In-flight work and human-control holds still keep a lease from expiring. +After daemon shutdown begins, no new lease-end teardown starts. +Shutdown waits for any lease-end teardown that already started before it tears sessions down itself. + +When a request closes a session, the daemon ends the lease of that session, unless the lease was allocated with `retainOnClose`. +The reply to that request reports each lease that it ended once, at response level, as `endedLeases: [{ leaseId, outcome: 'released' }]`. +On success, the report is in `data`. +On failure, the report is in `error.details`, including for a close that failed after the release. This applies to every request that can close a session: - `close`. - A `batch` close step. -- `close` in a `replay` or `test` script. -- Session cleanup after each `test` script. +- The `close` of a `replay` or `test` script. +- The session cleanup that `test` runs after each script. -Nested requests record on the outermost request's ledger. -Step replies never repeat that record. -If close finds its lease already inactive, it reports the lease as ended. +Nested requests record on the ledger of the outermost request, so a step reply never repeats the report. +If a close finds its lease already inactive, it reports that lease as ended. A client never sends a release after such a request. If a client stores the lease id, it forgets that id when the reply names it. A gateway that relays the reply reads the lease end from it. -Close and lease-end teardown use the same execution locks. -Whichever runs second finds the session gone. -Thus, close after lease end has no session and ends no lease. -If a client still stores the id, its next heartbeat finds an inactive lease. -It allocates a new lease with a fresh id. +Close and lease-end teardown take the same execution locks. +Thus, whichever runs second finds the session gone. +A close after a lease end has no session and ends no lease. +If a client still stores that lease id, its next heartbeat finds the lease inactive. +The client then allocates a new lease under a fresh id. ## Consequences Device contention can fail before platform execution with an explicit device-lease error. -The error includes backend, provider, selected device key, and owning lease expiry. +That error includes the backend, provider, selected device key, and owning lease expiry. Backend-only leases remain valid for older remote clients. -Provider-aware clients receive device-level contention and clearer recovery. +Provider-aware clients get device-level contention and clearer recovery. ## Admitted request work -Admission renews a lease once. -The request does not renew it again during execution. -Without preservation, a command longer than its inactivity TTL could expire its own device lease and lose its session before response. -Admitted work therefore preserves its lease like a human-control hold. -While the request remains wanted, it defers expiry. -If work finishes while still wanted, it renews the existing inactivity TTL from completion time. -After client disconnect, the request preserves nothing. -It neither defers expiry beyond cancellation nor renews the lease when work eventually finishes. -Thus, a handler that ignores cancellation cannot retain a rented device. - -The client's requested inactivity TTL determines which commands need preservation. +A lease renews when a request is admitted, and never again while that request runs. +Thus, an admitted command slower than the inactivity TTL used to expire the lease that paid for its own device. +The daemon then tore the session down while the client still waited for its result. +Admitted work therefore preserves its lease in the same way as a human-control hold does. +While the request is still wanted, it defers expiry. +If the request finishes while it is still wanted, it renews the lease for its existing inactivity TTL from the moment the work ended. +A request whose client hung up preserves nothing. +It neither defers expiry past that cancellation nor renews the lease when the request finally finishes. +Thus, a handler that ignores its cancellation cannot hold a rented device open. + +The inactivity TTL that the client asked for determines which leases this preservation reaches. The daemon default is one minute. -A cloud WebDriver connection profile requests ten minutes. -A single command longer than its lease is therefore ordinary with the default. -The longer window requires a profile. +A cloud WebDriver connection profile asks for ten minutes. +Thus, a single command that runs longer than its own lease is ordinary on the default. +On the longer window, such a command is only reachable through a profile. ## Provider allocation -Allocation is also admitted work. -The registry records a lease before a hosted provider creates its session. -Session creation can exceed the lease's inactivity TTL. -Without preservation, the client could receive an expired or nearly expired lease and leave a paid session orphaned. -Allocation therefore holds a work pass while the provider allocates. -The lease cannot expire during that work. -If successful allocation remains wanted, the inactivity TTL starts at completion. -A disconnected requester preserves nothing, and its allocation is released as before. +Allocation is admitted work too. +The registry records a lease before a hosted provider creates the session behind it. +Creation of that session can take longer than the inactivity TTL of the lease. +Thus, a lease timed from its record was already expired, or nearly expired, when its client first received it. +The paid session that the lease pointed at was orphaned. +The allocation therefore holds a work pass while the provider allocates. +The lease cannot expire during that allocation. +If a successful allocation is still wanted by its requester, the inactivity TTL starts from the moment allocation completed. +A requester that hung up preserves nothing, and its allocation is released as before. ## Client-side work that precedes admission -Admitted-work preservation does not cover earlier client work. -Artifact install uploads bytes before admission of the request that consumes them. -Without client protection, an upload longer than the inactivity window could expire the device lease (#2946). -The caller therefore sends lease heartbeats over the ordinary transport throughout that phase. -It names exactly the command's lease scope and supplies no payload of its own. -An install specifies no window, so its heartbeats renew the lease's existing window. -If a caller specifies a window, it continues to renew that window. - -Resolving an absent window to the registry default would shorten every lease allocated above that default. -Admission previously had the same defect through a proxy-specific default on each request. -Admission therefore also renews the existing lease window. -The lease retains the window named by its client at allocation. - -The first heartbeat starts with the phase, without a cadence delay. -It proves that the lease used by the upload remains alive. -Otherwise, a lease shorter than a fixed cadence could expire before renewal. -An already-lost lease is detected early, but not before phase start. -Hashing, preflight, and initial stream work can run while the first heartbeat remains outstanding. -Thus, loss becomes known during upload instead of afterward. -Each heartbeat reports the window it renewed. -The loop schedules its successor when the heartbeat starts, not when it completes. -A heartbeat without an answer is therefore abandoned on schedule without stopping the schedule. -A completed phase does not wait for an in-flight heartbeat. - -Cadence and budget intentionally differ. -A heartbeat starts every third of the window but may use the whole window for its answer. -A cadence-sized budget would always reject round trips longer than one third of the window. -This includes tunneled proxies and uplinks saturated by the upload. -Those slow links are exactly where heartbeat protection is needed. -The window defines how long continued liveness matters. -It therefore also defines the maximum useful wait for an answer. - -The actual ceiling is the shorter of the window and the heartbeat's request policy, currently ninety seconds. -Thus, a ten-minute lease permits a ninety-second answer budget. -The daemon renews the lease when it handles the request, before its answer. -A round trip abandoned at the cap has still extended the lease if the request reached the daemon. -If it never arrived, retaining a socket for the rest of a multi-minute window does not restore transport. -The cap refuses that idle wait. -It applies only to windows longer than the policy. -Those heartbeats already receive the longest answer deadline available to a lease. -Shorter windows use their own window as the bound. - -Each heartbeat is a fresh request, never a rewritten protected request. -Request identity controls cancellation of a timed-out heartbeat. -Heartbeats must not inherit each other's cancellation. -A heartbeat ends the phase with its error and cancels upload if it finds: - -- A lost lease. -- A missing scope. -- A scope owned by another lease. -- A daemon refusal of the heartbeat's own scope and window that an identical successor cannot correct. - -This prevents completed uploads for unowned devices or leases that cannot renew. -Any other heartbeat failure is reported without stopping the phase. -A later heartbeat can recover from one lost request. -This includes a request abandoned at its budget, so recovery applies to stalled round trips as well as immediate failures. +The protection of admitted work covers nothing that happens before a request is admitted. +An artifact install uploads the artifact from the caller before the request that will consume it is admitted. +Thus, an upload slower than the inactivity window of the lease expired the lease that paid for the target device of the bytes (#2946). +The caller therefore sends lease heartbeats over the ordinary transport for as long as that phase runs. +Each heartbeat names the lease scope exactly as the command named it, and carries no payload of its own. +An install names no window, so its heartbeats renew for the window that the lease already carries. +A caller that did name a window continues to renew on that window. + +A heartbeat used to resolve an absent window to the registry default instead. +That resolution shortened, without notice, every lease allocated above that default. +That was the other half of the reason that the upload could not survive. +Request admission had its own copy of that mistake. +It named a proxy-specific default for every admitted request. +Thus, a lease allocated longer than the default lost its window on the next command. +Admission also renews on the window of the lease. +The window that a lease carries is the one that its client named when it allocated. + +The first heartbeat is sent when the phase starts, not one cadence later. +The reason is that a heartbeat proves that the lease on which the upload spends time is still alive. +Otherwise, a lease shorter than any fixed cadence would lapse before anything renewed it. +A lease that is already gone is caught early, but not before the phase begins. +Hashing, the preflight, and the start of the stream can all run while the first heartbeat is still outstanding. +Thus, the first heartbeat makes sure that the loss is learned during the upload instead of after it. +Each heartbeat answers with the window that it just renewed. +The loop arms the next heartbeat when the current heartbeat starts, not when it settles. +Thus, a heartbeat that never answers is abandoned on schedule and does not stop the schedule. +A phase that settles does not wait for a heartbeat still in flight. + +Cadence and budget are deliberately two numbers. +A heartbeat is armed every third of the window, but it is allowed the whole window to answer. +If the budget were equal to the cadence, a heartbeat that needs more than a third of a window for its round trip could never answer. +Examples are a tunneled proxy and an uplink that the upload itself saturates. +Every heartbeat would then time out again on exactly the slow links that the heartbeat exists to protect. +The window, not the cadence, is the interval in which liveness still matters. +Thus, the window is also the longest time that an answer to a heartbeat is worth a wait. + +That ceiling is the shorter of the window and the heartbeat's own request policy, currently ninety seconds. +Thus, a ten-minute lease gives a heartbeat ninety seconds instead of the full window. +This limit costs nothing that the heartbeat exists to provide. +The daemon renews the lease when it handles the request, before it answers. +Thus, a round trip abandoned at the cap has still extended the lease if the request ever arrived. +If the request never arrived, the transport has stopped carrying traffic. +A socket held open for the rest of a window measured in minutes does not help that transport. +The cap refuses exactly that idle hold. +The cap binds only on windows longer than the policy. +On those windows, the heartbeat already has the longest answer deadline that any lease gets. +Shorter windows are bounded by their own window, as above. + +A heartbeat is a fresh request each time, never the protected request rewritten. +The reason is that a timed-out heartbeat is canceled under its request identity, and heartbeats must not inherit each other's cancellation. +In these cases, a heartbeat ends the phase with that error and cancels the upload: + +- The heartbeat finds that the lease is gone. +- The heartbeat finds that this request can never renew the lease. + This is so if the scope is missing or belongs to another lease. + This is also so if the daemon rejects the scope and window that the heartbeat itself was built with, which no later heartbeat will ask differently. + +Thus, the upload does not finish bytes against a device that nobody owns or a lease that will stop renewing. +A heartbeat that fails for any other reason is reported, and the phase continues. +The reason is that a later heartbeat covers one lost request. +This includes a request abandoned at its budget instead of answered. +Thus, that promise holds for a stalled round trip, not only for a round trip that fails fast. ## Human control Human-control holds coexist with an open remote session. -`LeaseRegistry` owns them. +They belong to `LeaseRegistry`. They use the same backend/provider/device contention key as device-aware leases. -The registry owns one lifecycle for hold heartbeat, expiry, lease preservation, and release refresh. -Release or expiry of the last hold restores the lease's existing inactivity TTL. -For expiry, renewal uses the hold's expiry instant. +Hold heartbeat, expiry, lease preservation, and release refresh are one registry-owned lifecycle. +Release or expiry of the last hold gives the lease its existing inactivity TTL again. +Expiry uses the hold's expiry instant. Tenant hold operations are ordinary daemon RPCs admitted through `request-admission.ts`. -Only the admitted lease supplies their device. +Their device comes only from the admitted lease. Host administration is a distinct loopback capability authenticated with the daemon token, never a tenant credential. Tenants cannot modify host holds. Mutation admission derives from existing recording effects, observation-class inventory, and observability semantics. -Takeover and lease heartbeats are exempt from the mutation fence, but retain ownership checks. -Unknown effects are mutations. -A pending activation fences new mutations. -Before it reports active, it drains mutations already admitted. +Takeover and lease heartbeats are exempt from the mutation fence, not from their ownership checks. +Unknown effects are treated as mutations. +A pending activation fences new mutations and drains the mutations already admitted before it reports active. Advisory execution locks alone do not establish this guarantee for fresh sessions. -Activation follows the calling RPC or host HTTP request's cancellation signal. -If disconnect occurs during drain, only that request's pending hold is removed. +Activation follows the cancellation signal of the calling RPC or host HTTP request. +A disconnect during the drain removes only the pending hold of that request. Successor and unrelated holds remain intact. -Canceled activation does not cancel mutations under drain. +A canceled activation does not cancel the mutations under drain. Completed holds use their TTL or explicit release lifecycle. -Holds and ordinary proxy leases remain in memory and do not survive daemon restart. +Holds and ordinary proxy leases are in-memory and do not survive daemon restart. Controllers must reconnect and establish them again. -No persisted hold store exists. +No persisted hold store is used. Local takeover is deferred. -A future host-global human-control fence must coexist with the local session's device claim instead of acquiring it exclusively. +A future host-global human-control fence must coexist with the device claim of the local session, not acquire it exclusively. ## macOS app leases A lease on the macOS host would rent the whole desktop. -The macOS platform therefore rents one app instead. -A `macos-app` lease uses a bundle id as its device key, optionally pinned to one process: `` or `@`. -If tenants could name this scope, they would choose their own confinement. -Thus, only a host administrator allocates these leases through loopback `/admin/leases` with the daemon token, as for host holds. -Tenant `lease_allocate` refuses this backend. -The host selects the lease id. -A repeated PUT renews it. -If a PUT names another scope for an existing id, it is refused instead of changing scope. +Thus, the macOS platform rents one app instead. +The device key of a `macos-app` lease is a bundle id, optionally pinned to one process (`` or `@`). +If a tenant could name the scope, the scope would be self-chosen. +Thus, only a host administrator allocates a `macos-app` lease, over the loopback `/admin/leases` route that uses the daemon token like host holds. +Tenant `lease_allocate` refuses the backend. +The host picks the lease id. +A repeated PUT renews the lease. +A PUT that names another scope for an existing id is refused, not rewritten. -Heartbeat, expiry, and restart loss follow ordinary lease behavior, with two exceptions: +Heartbeat, expiry, and the loss on daemon restart are those of any lease, with two exceptions: -- A tenant heartbeat or request cannot renew beyond the window of the host's last PUT. -- A tenant cannot release the lease. `lease_release` returns `MACOS_APP_LEASE_HOST_OWNED`. +- A tenant heartbeat or request cannot renew the lease for longer than the window of the host's last PUT. +- A tenant cannot release the lease. `lease_release` is refused with `MACOS_APP_LEASE_HOST_OWNED`. -The lease ends through: +The lease ends through one of these: -- Host `DELETE /admin/leases/`. +- The host's `DELETE /admin/leases/`. - Expiry. -- Daemon restart. -- Close of a session it holds, if the host set `retainOnClose` to false. +- The loss on daemon restart. +- The close of a session that the lease holds, if the host set `retainOnClose` to false. -Request admission confines every admitted request under the lease. -This includes `batch` steps and `replay` actions when they enter admission again. -Confinement requires: +Request admission confines every request admitted under the lease. +Thus, `batch` steps and `replay` actions are confined when they enter admission again. +The confinement has these parts: -- Commands whose registry descriptor declares `appLease: 'allowed'`. +- An allow list of commands: the commands whose command registry descriptor declares `appLease: 'allowed'`. Later commands are refused. - Among commands otherwise exempt from lease admission, only `lease_heartbeat` and `lease_release` declare it. -- `open` and `close` limited to the leased bundle. -- Only the `app` surface. -- Only window screenshots. + Of the commands that lease admission otherwise exempts, only `lease_heartbeat` and `lease_release` declare it. +- `open` and `close` of the leased bundle only. +- The `app` surface only. +- Window-only screenshots. - No input that names a host path or launches beside the app. -- An existing session for the leased app on every request except `open` and the `batch` envelope. +- For every request except `open` and the `batch` envelope, an existing session that is the leased app. -Thus, a request without a session cannot fall back to the host Mac. -`open` requires the native app backend from ADR 0031. -XCTest posts screen events that can reach outside the app window. -Before each admitted request, a pid-pinned lease is verified against the running process. -A session under the lease holds its app, not the Mac. +Thus, a request that names no session cannot fall back to the host Mac. +`open` requires the native app backend (ADR 0031), because XCTest posts screen events that can land outside the app's window. +Before each admitted request, a pid-pinned lease is checked against the running process. +A session opened under the lease holds its app, not the Mac. It takes no host device claim. Other app-leased sessions on the same Mac do not conflict with it. Thus, one daemon serves several leased apps beside the host's own sessions. These rules bind a request that names the lease or runs in its session. -Daemon policy `leases.require` in ADR 0029 refuses requests without a lease. -The proxy token is one credential shared by every client. -A host that serves several clients through one proxy therefore authenticates each client. -It sets tenant, session isolation, and lease on every forwarded request. +A daemon policy `leases.require` (ADR 0029) refuses requests that name no lease. +The proxy token is one credential for every client. +Thus, a host that serves several clients through one proxy authenticates each client. +The host sets the tenant, session isolation, and lease of that client on every request that it forwards. From 62470f5ddcd76f4c73166f798ac790077b17d8b0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:55 +0200 Subject: [PATCH 38/55] docs(adr): restore the frozen meaning in ADR 0008 Restore "As of 2026-07" on the descriptor-registry live list; the `read` alias it names was retired later in #1877. Restore the "Parity gate retired (2026-09)" label and its reason, and the ADR ownership sentence. Mark the import-cycle inversion as a migration risk while keeping the `Input`/`Result` ownership rule. Restore "every", "never collapse", "must preserve", "is bound by", and "the remaining deferred work". --- docs/adr/0008-command-descriptor-registry.md | 51 +++++++++++--------- 1 file changed, 29 insertions(+), 22 deletions(-) diff --git a/docs/adr/0008-command-descriptor-registry.md b/docs/adr/0008-command-descriptor-registry.md index b6f3e1eeb8..dd3b3165f0 100644 --- a/docs/adr/0008-command-descriptor-registry.md +++ b/docs/adr/0008-command-descriptor-registry.md @@ -7,10 +7,11 @@ Accepted Completed process history is in git history. > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** `CommandDescriptor` remains the root for command declarations. -> Device-command capability buckets evolve into typed required and preferred runtime use with leaf/device/provider-specific runtime facts. +> Device-command capability buckets evolve into typed required and preferred runtime use joined with leaf/device/provider-specific runtime facts. > Inventory commands declare inventory use instead of a fabricated device binding. > The descriptor's other derived projections remain accepted. > +> **Parity gate retired (2026-09).** The completed migration deleted every hand-authored table, so a parity test has no second table to compare. > No parity test exists. > Do not add or search for one. > Compile-time guards (`CommandOwnerFileClaimsAreComplete`) and literal pin-set tests now enforce totality. @@ -19,11 +20,11 @@ Completed process history is in git history. ## Rules at a glance - Compose one `CommandDescriptor` per command from facets owned by their domains. -- Derive consumer tables through pure projections. -- Keep shared narrow command traits on the descriptor. -- Keep client `invoke` and in-daemon `execute` distinct across the process boundary. +- Derive every consumer table through pure projection. +- Keep narrow command traits that affect multiple projections on the descriptor. +- Keep the cross-process client `invoke` seam and the in-daemon `execute` seam distinct. Never collapse the process boundary. - Preserve all four invariants from ADR 0003. -- The descriptor module must own `Input`/`Result` types. Derive the client as a view type under a lint boundary. +- The descriptor module must own the `Input`/`Result` types. The client must be derived as a view type, and a lint boundary enforces this. ## Context @@ -49,7 +50,7 @@ However, derivation stops at the command-surface boundary. Beyond it, tables remain manually maintained. ADR 0003 deliberately gives daemon route/policy a separately owned internal registry with a small predicate interface. -Its 2026-06 update defines four invariants for any single-declaration derivation model. +Its 2026-06 update sets four invariants that any single-declaration/derivation model must preserve. This ADR defines that model. ## Decision @@ -58,12 +59,12 @@ Introduce one `CommandDescriptor` per command. It **composes facets owned by their domains**. Every consumer table is **derived** through pure projection: -- The descriptor composes catalog identity, command-family surface projection hooks, `capability`, `daemon`, platform dispatch membership, response data transforms, and typed-result hooks. - `src/commands/**` retains CLI schema/reader and executable surface metadata ownership. - ADR 0003 defines the daemon facet's route and request-policy traits. +- The descriptor composes catalog identity, command-family surface projection hooks, a `capability` facet, a `daemon` facet, platform dispatch membership, response data transforms, and typed-result hooks. + `src/commands/**` remains the owner of the CLI schema/reader and executable surface metadata. + ADR 0003 shapes the route and request-policy traits of the `daemon` facet. - Narrow command traits that affect multiple projections belong to the descriptor. Interaction post-action observation (`--settle` / `--verify`) is one example. - CLI flags, command metadata, timeout policy, and tests derive from one fact instead of repeated command-name lists. + Thus, CLI flags, command metadata, timeout policy, and tests derive from one fact instead of repeated command-name lists. - These surfaces become pure projections of the descriptor set: - Public catalog. - Capability matrix. @@ -74,10 +75,10 @@ Every consumer table is **derived** through pure projection: - Descriptor-backed Node client surface. A total map keyed on the command-name union replaces the `src/core/dispatch.ts` `switch`. A missing handler therefore causes a compile error. -- Cross-process client `invoke` and in-daemon `execute` remain distinct. - The process boundary is never removed. +- The cross-process client `invoke` seam and the in-daemon `execute` seam remain distinct. + The process boundary is never collapsed. -This design follows all four ADR 0003 invariants: +This design composes with ADR 0003's four invariants and is bound by them: 1. Daemon-owned declaration, never inline in the public surface. 2. Unchanged predicate interface. @@ -87,8 +88,9 @@ This design follows all four ADR 0003 invariants: ## Alternatives Considered - **Keep manually synchronized tables.** This avoids migration risk. - However, each command still costs approximately 24 changed files, with alignment enforced only by convention and tests. -- **Use one flat public descriptor with inline daemon fields.** This puts daemon-only policy in the public command surface. + However, it is the status quo that this ADR exists to remove. + Each command costs approximately 24 changed files, with alignment enforced only by convention and tests. +- **Use one flat public descriptor with inline daemon fields.** This puts daemon-only policy back into the public command surface. ADR 0003 and its update explicitly forbid this ownership combination. - **Generate code at build time.** This is a valid option. However, runtime derivation with `as const satisfies` retains the source of truth in type-checked TypeScript. @@ -98,14 +100,15 @@ This design follows all four ADR 0003 invariants: A plain new command changes approximately 1–2 files. Per-platform behavior remains N implementations behind the descriptor's `execute`. -The descriptor is a prerequisite for typed per-command results in ADR 0010, which deletes the `src/client-types.ts` mirror. +The descriptor is the prerequisite for typed per-command results in ADR 0010, which deletes the `src/client-types.ts` mirror. It also supplies the capability facet used by the platform-plugin work in ADR 0009. -Import-cycle inversion is the principal risk. -`command-catalog.ts` has approximately 95 importers, and the family facet currently imports `AgentDeviceClient`. -Thus, the descriptor module must own `Input`/`Result` types. -The client must be derived as a view type, enforced by a lint boundary. -The descriptor registry supplies: +During the migration, the import-cycle inversion was the principal risk. +At that time, `command-catalog.ts` had approximately 95 importers, and the family facet imported `AgentDeviceClient`. +Thus, the descriptor module must own the `Input`/`Result` types. +The client must be derived as a view type, and a lint boundary enforces this. + +As of 2026-07, the descriptor registry is live for these surfaces: - Command identity: `PUBLIC_COMMANDS`, `INTERNAL_COMMANDS`, and local CLI names. - Daemon registry traits. @@ -121,6 +124,10 @@ Command-family surface metadata remains under `src/commands/**`, beside CLI gram Coherence checks compare it with the descriptor CLI catalog. Thus, new surface names cannot differ from the descriptor root. -Public Node-client typed-result narrowing remains deferred outside this registry migration in [#1153](https://github.com/callstack/agent-device/issues/1153). +The remaining deferred work is the public Node-client typed-result narrowing. +[#1153](https://github.com/callstack/agent-device/issues/1153) tracks this work outside this registry migration. Some commands still expose dynamic daemon response payloads or retain closed result contracts in runtime modules. Before their client methods can stop returning `CommandRequestResult`, those contracts must move. + +This ADR owns the decision and its constraints. +The descriptor projections enforce the delivered end state. From fe766245a20fb4ae282ae9db94f9dd8054be2736 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 39/55] docs(adr): restore the frozen meaning in ADR 0009 Restore "Implementation status as of 2026-08" with its Shipped / Decision-only support boundary / Retained compatibility / Deferred labels. Restore the command-atomic cutover wording and ADR 0019's term "legal injected substrate transition". Restore "no coverage is claimed" as a dated statement, not a ban, and the "registry and enforcement paths are authoritative" sentence. Fix a false causal clause for visionOS and restore base terms (OS-agnostic, net-new, seam, must not be flattened). --- docs/adr/0009-apple-platform-consolidation.md | 99 ++++++++++--------- 1 file changed, 54 insertions(+), 45 deletions(-) diff --git a/docs/adr/0009-apple-platform-consolidation.md b/docs/adr/0009-apple-platform-consolidation.md index 895732ff15..02c538c6d0 100644 --- a/docs/adr/0009-apple-platform-consolidation.md +++ b/docs/adr/0009-apple-platform-consolidation.md @@ -8,30 +8,32 @@ Completed process history is in git history. > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** The Apple family and explicit `AppleOS` leaf axis remain accepted. > The lazy platform-module registry and private `@agent-device/platform-*` packages supersede the shallow `PlatformPlugin` shape and root `src/platforms/**` location. -> Each execution migration covers one complete command. -> Shared physical mechanics move only through ADR 0019's permitted injected transition or after their last legacy consumer. +> Each command cuts over to the new execution path in one atomic step. +> Shared physical mechanics move only through ADR 0019's legal injected substrate transition or after their last legacy user. ## Rules at a glance - Use one `apple` Platform with an `AppleOS` discriminant, not six Platform literals. -- Keep shared Apple mechanics in `packages/platform-apple/src/core/` and per-OS code in `packages/platform-apple/src/os//`. +- Keep the OS-agnostic Apple engine in `packages/platform-apple/src/core/` and genuinely per-OS code in `packages/platform-apple/src/os//` leaves. - Express per-OS capability differences as data keyed by `AppleOS`. -- Preserve macOS's distinct AppKit leaf and tvOS's focus-only interaction contract. +- Keep macOS as a distinct AppKit leaf. +- Do not flatten the tvOS focus-only interaction contract across OSes. - Keep watchOS as an unsupported sentinel. Discovery never produces it. -- Keep public `ios`/`macos` output separate from internal family ownership. -- Do not claim public-command coverage for visionOS or watchOS. visionOS spatial-input QA remains deferred. +- Internal family ownership must not leak into the public `ios`/`macos` leaf projection. +- As of 2026-08, no public-command coverage is claimed for visionOS or watchOS. Net-new visionOS spatial-input QA is deferred. ## Context Apple support uses an asymmetric model. -An `ios` directory contains much of the shared Apple engine. -`Platform` has separate `ios` and `macos` literals, but tvOS uses `platform: 'ios' + target: 'tv'`. +Physically, Apple support is spread across an `ios` directory, and that directory is actually the shared Apple engine. +`Platform` has separate `ios` and `macos` literals. +However, tvOS is not a platform: it is `platform: 'ios' + target: 'tv'`. `resolveApplePlatformName` in `src/kernel/device.ts` reconstructs the OS name late and loses information. Approximately 697 LOC of macOS code resides in `src/platforms/ios/`. `src/platforms/macos/devices.ts` is a 19-line stub imported by iOS discovery, which reverses the intended dependency direction. The Apple interactor in `src/core/interactors/apple.ts` also imports from `platforms/ios`. -A survey by four investigators found approximately 85% of `platforms/ios` already OS-neutral: +A survey by four investigators found approximately 85% of `platforms/ios` already OS-agnostic: - Runner stack. - Tool-provider. @@ -49,9 +51,10 @@ iPadOS, visionOS, and watchOS have no model. Model Apple OSes with an **`AppleOS` discriminant** (`ios | ipados | tvos | watchos | visionos | macos`) under one `apple` Platform. Do not use six `Platform` literals. -Consolidate the OS-neutral Apple engine under `packages/platform-apple/src/core/`. +Consolidate the OS-agnostic Apple engine under `packages/platform-apple/src/core/`. Keep genuinely per-OS code in `packages/platform-apple/src/os//` leaves. The Apple plugin is the first instance of the platform-plugin registry. +The registry and enforcement paths named below are authoritative. Per-OS capability differences become data keyed by `AppleOS`. ## Alternatives Considered @@ -59,20 +62,23 @@ Per-OS capability differences become data keyed by `AppleOS`. - **Give each Apple OS its own `Platform` literal.** Rejected. `DeviceTarget` (`mobile | tv | desktop`) already spans platforms. Android TV uses `target: 'tv'`, so a `tvos` literal would conflict with the form-factor axis. - Approximately 15 `isApplePlatform` sites and 52 `macos` branches would need to enumerate six literals. + Approximately 15 `isApplePlatform` sites and approximately 52 `macos` branch sites would need to enumerate six literals. This would also break the working single-bucket `apple` capability/selector model. - **Keep the existing structure.** Rejected. It retains tvOS/macOS asymmetry, mislabeled macOS code, and lossy target→OS inference without a path to iPadOS/visionOS. - **Exclude macOS from consolidation.** Rejected. - macOS already builds through the same XCUITest project, and approximately 697 LOC resides in `platforms/ios`. + macOS is already entangled. + It builds through the same XCUITest project, and approximately 697 LOC of its code resides in `platforms/ios`. Exclusion would retain the incorrect label. - A distinct AppKit leaf normalizes ownership while preserving macOS-specific behavior. + A distinct AppKit leaf normalizes macOS without making it the same as the other OSes. ## Consequences -A first-class Apple OS needs only a leaf module and runner-profile row for the shared structure. -iOS/iPadOS/tvOS/macOS mostly require moves and renames because the engine does not need the specific Apple OS. -visionOS requires scoped new work because XCUITest supports it: +Adding a first-class Apple OS becomes cheap: it needs a leaf module and a runner-profile row. +iOS/iPadOS/tvOS/macOS mostly require moves and renames because the engine never needed to know which Apple OS it drives. +visionOS is scoped net-new work. +XCUITest supports visionOS. +The work includes: - A profile row. - A build case. @@ -80,40 +86,43 @@ visionOS requires scoped new work because XCUITest supports it: - A wider discovery filter. - Real spatial-input QA. -watchOS is an explicit **unsupported sentinel** because XCUITest cannot operate its UI. -macOS remains a distinct AppKit leaf with its helper binary and menubar/desktop surface model. -The tvOS focus-only interaction contract, with no coordinate `tap`, must remain distinct across OSes. +watchOS is an explicit **unsupported sentinel** because XCUITest cannot drive watchOS UI. +macOS remains a distinct AppKit leaf, and its helper binary and menubar/desktop surface model are preserved. +The tvOS focus-only interaction contract, with no coordinate `tap`, must not be flattened across OSes. Snapshot fidelity is uneven. The deep-RN AX-server fallback is iOS-simulator-only. Public leaf output remains a separate compatibility projection. This design composes with ADR 0008's descriptor capability facet and ADR 0003. -The support boundary includes: - -- Internal `Platform` ownership uses `apple`, with additive `appleOs`. -- The shared Apple engine resides under `packages/platform-apple/src/core`. -- macOS leaf files reside under `packages/platform-apple/src/os/macos`. -- A dedicated tvOS leaf resides under `packages/platform-apple/src/os/tvos`. -- Internal imports access Apple modules directly. -- Per-`AppleOS` runtime facts include `packages/platform-apple/src/gesture-facts.ts`. - The former capability projection's predicates now belong to request-bound facts. -- `AppleOS` reserves watchOS, but the interactor rejects it. - `isSupportedAppleDeploymentLeaf`, the Apple interactor, and gesture admission all reject watchOS. - Discovery never produces it. -- visionOS has profile/build/discovery support and simulator-deployment evidence. - -visionOS discovery and simulator deployment are supported and unit-tested in `packages/platform-apple/src/inventory-classification.ts` and `packages/platform-apple/src/deployment/runtime.test.ts`. -CoreDevice-backed physical devices also admit app deployment. -XCTest-backed physical deployment remains unsupported. -Push remains simulator-only. -No public-command coverage is claimed for visionOS or watchOS. -This ADR records that decision-only boundary without a command-coverage manifest for either leaf. - -The public wire retains `ios`/`macos` leaves through `PUBLIC_PLATFORMS`. -Internal family ownership must not enter that projection. -New visionOS spatial-input QA remains deferred. +Implementation status as of 2026-08: + +- Shipped: + - The internal `Platform` collapse to `apple`. + - Additive `appleOs` groundwork. + - The shared Apple engine under `packages/platform-apple/src/core`. + - macOS leaf files under `packages/platform-apple/src/os/macos`. + - A dedicated tvOS leaf under `packages/platform-apple/src/os/tvos`. + - Direct internal imports to the Apple modules. + - The per-`AppleOS` runtime facts, including `packages/platform-apple/src/gesture-facts.ts`. + The former capability projection was retired after its predicates moved into request-bound facts. + - The watchOS **unsupported sentinel**. + The `AppleOS` type reserves watchOS, and the interactor rejects it. + XCUITest cannot drive watchOS UI. + `isSupportedAppleDeploymentLeaf`, the Apple interactor, and gesture admission reject it. + Discovery never produces it. + - visionOS profile/build/discovery, plus simulator-deployment evidence. +- Decision-only support boundary: + - visionOS discovery and simulator deployment are supported and unit-tested in `packages/platform-apple/src/inventory-classification.ts` and `packages/platform-apple/src/deployment/runtime.test.ts`. + - App deployment is also admitted for CoreDevice-backed physical devices. + - XCTest-backed physical deployment is unsupported. + - Push remains simulator-only. + - No public-command coverage is claimed for visionOS or watchOS. + - This ADR records this boundary without a command-coverage manifest for either leaf. +- Retained compatibility: the public wire still emits `ios`/`macos` leaves through `PUBLIC_PLATFORMS`. + Internal family ownership must not leak into that projection. +- Deferred: net-new visionOS spatial-input QA. This ADR owns the Apple family/leaf decision. -ADR 0019 owns the staged package/runtime boundary. -New visionOS spatial-input QA remains a separate follow-up outside consolidation. +ADR 0019 now owns the staged package/runtime seam. +Net-new visionOS spatial-input QA remains a separate follow-up outside the consolidation. From 4901940494cf994c6a5c3d1b26c376a638542246 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 40/55] docs(adr): restore the frozen meaning in ADR 0010 Restore the hint-required condition in the ADR 0010 glance, and use "pin" for help-text guarantees. Restore the scope "a user or agent can reach", "consumer surface", "most common agent failures", and "some consumers". Restore the condition on the ADR 0019 cleanup sentence, and keep the code-selection list from reading as a ban on the correct codes. Replace "subclasses" with the base term "sub-classification". --- docs/adr/0010-error-system.md | 109 +++++++++++++++++----------------- 1 file changed, 55 insertions(+), 54 deletions(-) diff --git a/docs/adr/0010-error-system.md b/docs/adr/0010-error-system.md index a5e4ef807e..04fe858d7d 100644 --- a/docs/adr/0010-error-system.md +++ b/docs/adr/0010-error-system.md @@ -5,94 +5,95 @@ > **Amended by [ADR 0019](0019-request-bound-platform-runtime.md).** Existing normalization and cause-preservation rules remain accepted. > If an operation and binding/resource cleanup both fail, the operation remains primary. -> Cleanup becomes structured secondary diagnostic evidence. -> A cleanup-only failure returns normally as a failure. +> In that case, the cleanup failure is structured secondary diagnostic evidence. +> If only cleanup fails, the cleanup failure surfaces normally. ## Rules at a glance -- Every reachable thrown error is an `AppError`. Reserve bare `Error` for provably unreachable invariants. -- For input validation, use `INVALID_ARGS`. For unknown caught errors, use `asAppError(err, fallbackCode)`. -- Use the most specific `KnownAppErrorCode`. Put machine-dispatchable subclasses in `details.reason`. -- Supply a hint only when the default would mislead. Preserve existing hints and causes through wrappers. -- Every consumer must carry at least code, message, and hint. Rehydration helpers copy all error fields. -- Failed requests always flush diagnostics. Messages must not contain raw stderr dumps or secrets. +- Every thrown error that a user or agent can reach is an `AppError`. Reserve bare `Error` for provably unreachable invariants. +- For input validation, throw `AppError('INVALID_ARGS', ...)`. To coerce an unknown caught error, use `asAppError(err, fallbackCode)`. +- Use the most specific `KnownAppErrorCode`. Put machine-dispatchable sub-classification in `details.reason`. +- If the per-code default hint would mislead, supply a hint. If the default suffices, omit the hint. +- When you wrap an error, preserve an existing hint and the cause chain. +- Every consumer surface must carry at least code, message, and hint. Rehydration helpers copy all fields. +- Failed requests always flush diagnostics. Messages must not embed raw stderr dumps or secrets. ## Context -`packages/kernel/src/errors.ts` centralizes `AppError`, `normalizeError`, `defaultHintForCode`, and `retriableForErrorCode`. -However, approximately 1,200 sites construct errors across CLI, daemon, and platform layers. -A July 2026 audit combined code examination with iOS/Android use. -It found a sound kernel but inconsistent call sites: +The error system is centralized in `packages/kernel/src/errors.ts` (`AppError`, `normalizeError`, `defaultHintForCode`, `retriableForErrorCode`). +However, approximately 1,200 construction sites exist across the CLI, daemon, and platform layers. +A July 2026 audit combined a code sweep with iOS/Android dogfooding. +It found the kernel sound, but call-site quality had drifted: -- Internal validation threw bare `Error`, which appeared as `UNKNOWN` with a misleading hint. -- Some codes had incorrect semantics, such as `COMMAND_FAILED` for busy devices or `INVALID_ARGS` for expired server-side resources. -- Common selector/ref failures lacked hints. -- Consumers lost fields, such as MCP tool errors that carried only `message`. +- Internal validation threw bare `Error`, which surfaced as `UNKNOWN` with a misleading hint. +- Some codes were semantically wrong, such as `COMMAND_FAILED` for busy devices and `INVALID_ARGS` for expired server-side resources. +- The most common agent failures, selector/ref misses, had no hints. +- Some consumers dropped fields; for example, MCP tool errors carried only `message`. -Without documented conventions, each new site selected its own behavior. +Nothing documented the conventions, so each new site decided them again. ## Decision 1. **Every thrown error a user or agent can reach is an `AppError`.** - Reserve bare `throw new Error(...)` for provably unreachable invariants. - Input validation, including daemon-side command input decode, throws `AppError('INVALID_ARGS', ...)`. - For unknown caught errors, use `asAppError(err, fallbackCode)` to preserve the cause chain. - Do not construct a replacement through `err instanceof AppError ? err : new AppError(...)`. + Bare `throw new Error(...)` is reserved for provably unreachable invariants. + Input validation, including daemon-side decoding of command input, throws `AppError('INVALID_ARGS', ...)`. + To coerce an unknown caught error, use `asAppError(err, fallbackCode)`, which preserves the cause chain. + Do not hand-roll `err instanceof AppError ? err : new AppError(...)`. 2. **Code selection.** Use the most specific `KnownAppErrorCode`. - `COMMAND_FAILED` means a genuine runtime failure of a well-formed request. - Never use it as a general code for: - - Capability gaps: `UNSUPPORTED_OPERATION`. - - Contention: `DEVICE_IN_USE`. - - Ambiguity: `AMBIGUOUS_MATCH`. - Only `DEVICE_IN_USE` is retriable by default. - The cross-worktree device-claim path overrides it with `retriable: false`. + `COMMAND_FAILED` is for genuine runtime failures of a well-formed request. + Never use `COMMAND_FAILED` as a catch-all for these cases: + - Capability gaps. Use `UNSUPPORTED_OPERATION`. + - Contention. Use `DEVICE_IN_USE`. + - Ambiguity. Use `AMBIGUOUS_MATCH`. + `DEVICE_IN_USE` is the only code that is retriable by default. + The cross-worktree device-claim path overrides that default to `retriable: false`. Add new codes to the union deliberately. - Put machine-dispatchable subclasses in `details.reason`, as the lease registry does. + Machine-dispatchable sub-classification goes in `details.reason`; the lease registry is the model. 3. **Hints answer "what should the agent run next".** If the per-code default from `defaultHintForCode` would mislead, a hint is required. If the default suffices, omit the hint. - Repeated boilerplate hints are worse than the default. - Shared failure modes use shared hint constants beside their detection code, instead of copied strings: + Boilerplate hints added in bulk are worse than the default. + Shared failure modes get shared hint constants next to the code that detects them, not copy-pasted strings: - `selectorFailureHint` and `STALE_REF_HINT` in `packages/selectors/src/internal/resolve.ts`. - `resolveIosDevicectlHint`. - `bootFailureHint`. - Wrappers preserve an existing hint instead of replacing it. -4. **Wrapping external tool failures.** Prefer unchanged errors from `exec.ts`. - A manual wrapper of an `allowFailure` result must carry `{ stdout, stderr, exitCode, processExitError: true }`. - Thus, `normalizeError` can expose the first meaningful stderr line instead of "tool exited with code N". + A re-wrap preserves an existing hint instead of overwriting it. +4. **Wrapping external tool failures.** Prefer `exec.ts` errors as they are. + A hand-rolled wrap of an `allowFailure` result must carry `{ stdout, stderr, exitCode, processExitError: true }`. + Thus, `normalizeError` can surface the first meaningful stderr line instead of "tool exited with code N". 5. **The wire error contract** contains: - `code`. - `message`. - `hint`. - - Redacted `details`. + - `details` (redacted). - `diagnosticId`. - `logPath`. - Optional typed signals `retriable` / `supportedOn`. - Unless confidently known, typed signals are absent. + The typed signals are absent unless they are confidently known. Every consumer surface must carry at least code, message, and hint: - CLI human output. - CLI `--json`. - SDK. - MCP. - Events timeline. - Rehydration helpers `throwDaemonError` and `toDaemonHttpRpcError` copy all fields. - `NormalizedError` exposes typed signals at its top level for `--json` consumers. + Rehydration helpers (`throwDaemonError`, `toDaemonHttpRpcError`) copy all fields. + `NormalizedError` hoists the typed signals, so `--json` consumers see them. 6. **Observability.** Failed requests always flush diagnostics. - Every error carries `diagnosticId` and ndjson `logPath`. - Daemon diagnostics reside under the session state dir. - Client diagnostics reside under `~/.agent-device/logs`. - Messages must not contain raw stderr dumps or secrets. - Structured context belongs in `details`, which is redacted at normalization/write time. + A `diagnosticId` and an ndjson `logPath` accompany every error. + Daemon-side diagnostics are under the session state dir. + Client-side diagnostics are under `~/.agent-device/logs`. + Messages must not embed raw stderr dumps or secrets. + Structured context belongs in `details`, which is redacted at normalize/write time. ## Consequences -- Agents can select behavior by `code`, retry by `retriable`, and follow `hint` without prose parsing. - Wording can improve without breaking consumers, except for strings fixed by help-text guarantees. - When those strings change, update their tests together. -- The known-code union remains small and meaningful. - The wider `(string & {})` type still passes runner/daemon codes through, so SDK consumers retain a `default` branch. -- New call sites have documented requirements: - - Correct code. - - Contextual message that names the target. - - Hint only when the default misleads. - - Preserved cause. +- Agents can branch on `code`, retry on `retriable`, and follow `hint` without prose parsing. + Wording can improve without breaking consumers, except for strings that help-text guarantees pin. + If you change one of those strings, update its tests in the same change. +- The known-code union stays small and meaningful. + The widened `(string & {})` type still lets runner/daemon codes pass through, so SDK consumers keep a `default` branch. +- New call sites have a documented bar: + - The correct code. + - A contextual message that names the target. + - A hint only when the default misleads. + - A preserved cause. From 0c3bf0a2cbbaceabc3c78462d5bbd11a78095831 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 41/55] docs(adr): restore the frozen meaning in ADR 0011 Restore ADR 0011 Migration plan step 3 as open: no fake runner reads tap-point-policy.json yet. Steps 1, 2, 4, and 5 stay deleted as complete. Restore the dated status qualifier (implemented through Layer 3, 2026-07-04). Restore both collapse conditions in the glance, name `RUNTIME_OPERATION_EFFECTS`, and restore "Remaining gaps". Restore dropped conditions, "only"/"merely"/"deliberately", the causal links, and the base terms ("class of bug", "ladder", "Swift twin", "hand-rolled"). --- .../0011-interaction-guarantee-contract.md | 476 +++++++++--------- 1 file changed, 247 insertions(+), 229 deletions(-) diff --git a/docs/adr/0011-interaction-guarantee-contract.md b/docs/adr/0011-interaction-guarantee-contract.md index 9f39d394bb..810dd0ad0e 100644 --- a/docs/adr/0011-interaction-guarantee-contract.md +++ b/docs/adr/0011-interaction-guarantee-contract.md @@ -2,69 +2,70 @@ ## Status -Accepted +Accepted (implemented through Layer 3, 2026-07-04: #1080, #1082–#1086, #1091, #1092; ambiguity contract amended 2026-08-07) Completed process history is in git history. ## Rules at a glance - Classify every dispatch-path × guarantee cell in the typed registry. -- Link each gap waiver to a tracking issue. Keep command-scoped claims within the path's command set. -- Give each TypeScript rule one shared implementation. Prove runner-side equivalents through golden fixtures. +- Link each `gap:` waiver to a tracking issue. Keep command-scoped claims within the commands that the path dispatches. +- Give each guarantee exactly one TypeScript home that all TypeScript paths import. Prove the parity of runner-side Swift twins through golden fixture tables. - Prove every non-waived cell through a tagged contract scenario. - Delegation-on-error does not prove success-path parity. -- Before native-ref execution, apply shared guards to the stored node. -- Never resend an Apple mutation that may have run. -- Distinct mutating selector subtrees fail with `AMBIGUOUS_MATCH`, except for the stated compatibility path. -- Keep synthesized coordinate gesture policy separate from element-targeting guarantees. +- On the native-ref path, before `tapTarget`/`fillTarget`, check offscreen, occlusion, and non-hittable against the node from the daemon snapshot. +- The Apple runner does not resend a mutating command whose first send may have run. +- Multiple mutating selector matches collapse only if they form one ancestor–descendant chain and every member resolves to the same actionable node. Otherwise, the mutation fails with `AMBIGUOUS_MATCH`. Maestro's explicit expected-point/non-hittable compatibility path stays separate. +- Keep synthesized iOS gesture policy outside the element dispatch-path matrix. ## Context Interaction commands reach the device through several dispatch paths. -These commands include `press`/`click`/`fill`/`longpress` and related read/wait surfaces that share resolution machinery. -Each path independently selected its latency and semantic compromises: +These commands are `press`/`click`/`fill`/`longpress` and, by extension, the read/wait surfaces built on the same resolution machinery. +Each path independently selected its latency-vs-semantics trade-offs: | Path | What it is | Why it exists | | --- | --- | --- | | `runtime-selector` | daemon tree capture → `resolveSelectorChain` → guards → coordinate tap | full semantics | | `runtime-ref` | session snapshot → ref lookup → guards → coordinate tap | full semantics | | `target-drag` | independently resolve source + destination → guards → one pointer plan | dual-endpoint drag semantics | -| `direct-ios-selector` | selector sent to the XCTest runner for native query and tap | avoids a full snapshot round trip | +| `direct-ios-selector` | selector sent to the XCTest runner, which queries and taps natively | avoids a full snapshot round trip | | `native-ref` | `backend.tapTarget`/`fillTarget` for `click @ref` / `fill @ref` | avoids resolution round trips | -| `coordinate` | raw x/y tap | explicit minimal semantics | +| `coordinate` | raw x/y tap | escape hatch; semantics intentionally minimal | | replay/maestro variants | `allowNonHittableCoordinateFallback`, replay-heal | third-party compatibility | -During one week, every shipped or detected interaction bug had the same cause. -One path enforced a guarantee that another path silently omitted: +Over one week of fixes, every interaction bug that we shipped or caught had the same pattern. +One path enforced a guarantee, and a sibling path silently did not have it: -- Offscreen refusal covered `@ref` targets but omitted selector targets and native runner taps. - A closed Bluesky drawer reported success as `Tapped (-161, 265)` (#1075). +- Off-screen refusal existed for `@ref` targets, but not for selector targets and not in the runner's native tap. + A closed Bluesky drawer reported `Tapped (-161, 265)` as success. #1075 fixed this. - `press @ref` attached `--verify` evidence. - The manually constructed response for `fill @ref` omitted it (#1064 review). -- adb failure classification enriched thrown errors but omitted errors from `allowFailure` results. - It wrapped `exec` but omitted semantic provider methods (#1067 review). -- The user's wait budget bounded prepare/replay/snapshot envelopes but omitted `wait` itself. - A `wait` timeout reset the daemon, while the same failure in `snapshot` preserved it (#1075). + The hand-built response branch of `fill @ref` dropped it (#1064 review). +- adb failure classification enriched thrown errors, but not errors from `allowFailure` results. + The classification wrapped `exec`, but not semantic provider methods (#1067 review). +- The user's wait budget bounded prepare/replay/snapshot request envelopes, but not `wait` itself. + Also, `wait` was on the daemon-reset timeout path, while `snapshot`, with the same failure mode, preserved the daemon (#1075). -The guarantees are small and well implemented. -The missing structure enumerates every path and every guarantee to detect an unclassified cell. -Reviews otherwise depend on simultaneous knowledge of both axes. -Live use detects the gaps only after incidents. +The guarantees themselves are small and well implemented. +The missing part is a structure that knows the full set of paths and the full set of guarantees, and can therefore detect an unfilled cell. +Reviews find such a cell only if a reviewer happens to know both axes at the same time. +Dogfooding finds such cells one production incident at a time. ## Decision -Make the path × guarantee matrix a machine-checked artifact with three enforcement layers. -Types enforce declaration **completeness**. +Make the path × guarantee matrix a first-class, machine-checked artifact with three enforcement layers. +Types enforce **completeness** of declarations. Shared implementations prevent **drift**. Generated test coverage enforces **truth**. ### Layer 1 — Declare: the matrix as a typed registry with a gate -Layer 1 proves **honesty/completeness**, not behavior. -It proves that every path declares a position on every guarantee and that referenced implementations exist. -Behavioral parity additionally requires Layer 2 fixture tables and Layer 3 contract scenarios. -Layer 1 alone prevents an undeclared cell. -An unproven cell remains a visible claim with an owner. +Layer 1 is an **honesty/completeness gate, not a truth gate**. +It proves that every path declares a stance on every guarantee and that the referenced implementations exist. +It does not prove the declared behavior. +Behavioral parity starts only after the fixture tables (Layer 2) and the contract scenarios (Layer 3) are in place. +Layer 1 alone still changes the failure mode: an unwatched cell becomes impossible. +An unproven cell is at least a visible claim that has an owner. `packages/contracts/src/interaction-guarantees.ts` declares both axes and requires classification of every cell: @@ -81,21 +82,22 @@ export const INTERACTION_GUARANTEES = [ ] as const; ``` -When a guarantee applies to only some path commands, cells may use command-scoped `appliesTo`. -For example, `verifyEvidence` applies to `press`/`click`/`fill`, but not `longpress`. +If a guarantee exists only for a subset of the path's commands, cells may be **command-scoped** through `appliesTo`. +For example, `verifyEvidence` applies to `press`/`click`/`fill`, but not to `longpress`. A path-wide claim would overstate coverage. -The gate rejects commands outside the path and redundant lists that cover all path commands. +The gate rejects an `appliesTo` entry that names a command that the path does not dispatch. +The gate also rejects a redundant `appliesTo` list that covers all commands of the path. -`responseConstruction` and `responseIdentity` remain separate. -A shared construction site and available identity fields need different completion strategies. -The former requires one Layer-2 refactor. -The latter requires per-path capability work. -`errorTaxonomy` is expected to split similarly later: +`responseConstruction` and `responseIdentity` are deliberately separate. +"Use one shared construction site" and "which identity fields this path can provide" have different closure strategies. +The former is a single Layer-2 refactor. +The latter is per-path capability work. +`errorTaxonomy` is expected to split the same way later: - Stable codes and fallback classification. - Rich selector diagnostics and hints. -Direct runner paths can implement codes before full diagnostics. +The reason is that direct runner paths can close codes long before full diagnostics. ```ts @@ -116,25 +118,28 @@ export const INTERACTION_DISPATCH_PATHS: Record< ``` `guarantees` is a `Record` over the guarantee union. -Thus, **tsc fails** if a new guarantee lacks classification for every path or a new path lacks any guarantee classification. -The unit gate `interaction-guarantees.test.ts` also verifies: +Thus, **tsc fails** if someone adds a guarantee without a classification for every path. +tsc also fails if someone adds a path without a classification of every guarantee. +The unit gate `interaction-guarantees.test.ts` also checks that: -- Every `via` string resolves to a real exported symbol. +- Every `via` string resolves to a real exported symbol, so declarations cannot become fiction. - Every `parityTable` names an existing fixture file. - Every `waived` reason is non-empty. -- Every `gap:` waiver has a `trackingIssue`. +- Every `gap:` waiver carries a `trackingIssue`. -Waivers require ownership as well as visibility. -One umbrella issue is sufficient, with sub-issues added when work is scheduled. -The gate enforces the link, not issue granularity. +Waivers must have an owner, not only visibility. +One umbrella tracking issue is sufficient, with sub-issues split off when work is scheduled. +The gate enforces the link, not the granularity. -The repository already uses this explicit-gap pattern in `scripts/integration-progress-model.ts` and ADR 0009's cross-command apple-leak guard. -The progress model detected the unclassified `--verify` flag on #1064. +This repository already proved the same "make the gap declare itself" pattern in two places: + +- `scripts/integration-progress-model.ts`, which caught the unclassified `--verify` flag on #1064. +- The cross-command apple-leak guard from the platform consolidation (ADR 0009). ### Layer 2 — Share: one implementation per rule, on both sides of the wire -Each guarantee has exactly one owner imported by every TypeScript path. -Existing owners include: +Each guarantee has exactly one home that all TypeScript paths import. +Most of these homes already exist after #1075: - `isNodeVisibleOnScreen`. - `accumulateDisambiguationCandidate`. @@ -143,68 +148,71 @@ Existing owners include: - `interactionResultExtra`. - `reconcileNonHittableHintWithEvidence`. -The registry's `via` fields identify them. -Fallow's duplication gate prevents new duplicate implementations. +The registry's `via` fields point at them. +Fallow's duplication gate prevents re-implementations from coming back. -Rules that must execute runner-side use pure-function Swift equivalents over plain geometry, without `XCUIElement`. -The direct iOS path cannot see the daemon tree. +Some rules must run **runner-side**, because the direct iOS path cannot see the daemon tree. +These rules get pure-function Swift twins that operate on plain geometry, without `XCUIElement`. `TapPointPolicy.isAllowed(elementFrame:windowFrame:)` is one example. -**Golden fixture tables** under `contracts/fixtures/` enforce parity through three consumers: +**Golden fixture tables** enforce parity. +These tables are JSON files under `contracts/fixtures/`, and three test suites consume them: 1. Vitest asserts the TypeScript rule against the table. -2. Runner Swift unit tests assert the Swift equivalent against the same table. +2. Runner Swift unit tests assert the Swift twin against the same table. CI already compiles them through "Swift Runner Unit Compile". 3. The provider harness's fake runner derives its behavior from the table. Integration scenarios therefore exercise the real contract instead of a manual approximation. -A TypeScript or Swift difference fails CI on the changed side without a simulator. +Then, a TypeScript or Swift drift turns CI red on the side that changed, without a simulator. -`contracts/fixtures/dispatch-disclosure.json` defines `AppErrorDetails.dispatched` on interaction failures. -Each producer event has one row with its required value: +`contracts/fixtures/dispatch-disclosure.json` is the table for `AppErrorDetails.dispatched` on interaction failures. +It has one row for each producer event, and each row has the value that the producer must leave. +The field has two values: - `no`: the operation never reached the device, or the command is a read, so a resend is safe. -- `unknown`: the operation may have reached the device, so observe before resend. +- `unknown`: the operation may have reached the device, so observe before a resend. -Only pre-dispatch refusals from target resolution, admission, or runner pre-send checks may report `no` for an operation. -No producer can prove execution on its failure path, so no third value exists. +Only a producer that refuses before dispatch (target resolution, admission, a runner pre-send refusal) may report `no`. +No producer can prove execution on its failure path, so there is no third value. Each request owns one dispatch ledger. -Every bound runtime operation declares its effect once in `RUNTIME_OPERATION_EFFECTS`, a record over runtime operation keys. -An operation cannot bind without an effect. -When a send returns, the request binding records each `mutates` operation in that request's ledger. -Thus, no bound runtime operation sends an unrecorded mutation. -Android ANR and blocking-dialog recovery remain outside the ledger. -Their `AndroidObservationAdapter.tap` and `openApp` calls reach the device without a record. -A nested request has its own ledger and transfers its sends to its parent's ledger. -Examples include batch steps, replay actions, and delegated `find` clicks or fills. - -The request router adds disclosure around every routed command. -After a sent mutation enters the ledger, any failure reports `unknown` and the sent count in `details.dispatchedSteps`. -This overrides both producer output and command declarations. -An incorrect `no` could cause a consumer to repeat an action that ran. -An incorrect `unknown` costs only an observation. -Before any sent mutation, registry `recordingEffect` controls disclosure: - -- For `'mutates-app'`, preserve the producer's value. - If no producer classified the failure, supply `unknown`. -- For `'observes-app'`, set `no` over any producer value. -- Without a declared effect, preserve the response unchanged. - -For reads, `no` means a resend repeats no app-visible action. -The registry declares `record`, `trace`, and `perf` as `'observes-app'`, although recorder/profiler controls reach the device. -No registry trait separates pure reads from device controls. -Those operations are `repeatable` because the device refuses or ignores a repeat. -If one bound operation performs several inputs, its producer counts them in `details.dispatchedSteps` because the ledger sees only one operation. -Examples include an Android double tap and an iOS press series. - -Each fixture row's id prefix identifies its driver file. -That file exercises the real producer. - -The Apple runner does not resend a mutation whose first send may have run. -After restart, it resends only a read-only command or a command that the first attempt provably did not write. -If a mutation's reply remains lost, failure carries `reason: runner_reply_lost` and `dispatched: unknown`. -This applies if the runner dies mid-command or status recovery finds neither a retained result nor a runner answer. -The fixture rows are: +Every bound runtime operation declares its effect once, in `RUNTIME_OPERATION_EFFECTS`, a record over the runtime operation keys. +Thus, an operation cannot bind without an effect. +When the send of a `mutates` operation returns, the request binding records the operation in the request's ledger. +Thus, no bound runtime operation sends a mutation outside the ledger. +Android ANR and blocking-dialog recovery is outside the ledger. +Its `AndroidObservationAdapter.tap` and `openApp` calls reach the device without a record. +A nested request records into a ledger of its own and moves its sends into its parent's ledger. +A batch step, a replay action, and a delegated `find` click or fill are nested requests. + +The request router discloses around every routed command. +After the ledger holds a sent mutation, a failure is `unknown`, with the sent count in `details.dispatchedSteps`. +This applies whatever produced the failure and whatever the command declares. +The reason: a wrong `no` makes a consumer resend an action that already ran, while a wrong `unknown` only costs an observation. +Before the ledger holds a sent mutation, the registry's `recordingEffect` decides: + +- For `'mutates-app'`, the router keeps a producer's value. + If no producer classified the failure, the router fills `unknown`. +- For `'observes-app'`, the router sets `no` over any producer value. +- A command with no declared effect passes through. + +For a read, `no` means that a resend repeats no app-visible action. +The registry declares `record`, `trace`, and `perf` as `'observes-app'`, although their recorder and profiler controls reach the device. +There are two reasons: + +- No registry trait separates a pure read from a device control. +- `RUNTIME_OPERATION_EFFECTS` declares those controls `repeatable` at the operation level, because the device refuses or ignores a repeat. + +A producer can run several device inputs inside one bound operation, such as an Android double tap or an iOS press series. +That producer counts the inputs in `details.dispatchedSteps` itself, because the ledger sees one operation. + +Each row names its driver file by id prefix, and that file drives the real producer. + +The Apple runner does not resend a mutating command whose first send may have run. +A restart resends only a command that the first attempt provably did not write, or a read-only command. +A mutation reply can stay lost: the runner dies mid-command, or status recovery finds neither a retained result nor a runner answer. +In that case, the mutation fails with `reason: runner_reply_lost` and `dispatched: unknown`. +The rows for this case are: - `ios-runner.transport.written-then-lost`. - `ios-runner.status.completed-without-retained-reply`. @@ -214,68 +222,63 @@ The fixture rows are: - `ios-runner.status.probe-failed`. - `ios-runner.status.unavailable`. -The `ios-runner.status.failed*` rows instead carry the runner's own answer. -A read retains its transport error and is resent. +The `ios-runner.status.failed*` rows carry the runner's own answer instead. +A read keeps its transport error and is resent. -An evidence gap remains before the router's locked scope. -Failures in session resolution, lock acquisition, lease admission, or daemon-policy admission never reach disclosure and carry no `dispatched`. -They send nothing, but consumers must interpret absence as `unknown`. +Remaining gaps: a failure before the router's locked scope (session resolution, lock acquisition, lease and daemon-policy admission) never reaches the disclosure and carries no `dispatched`. +Such a failure sends nothing, but a consumer must read the absent field as `unknown`. -`runBatch` reports `no` only if every executed step declares `observes-app`. -Any other executed step makes the batch failure `unknown`, including steps without `recordingEffect`. -This applies to every caller, including callers with their own `invoke`. +`runBatch` reports `no` only if every executed step is a command declared `observes-app` (a read). +Any other executed step makes the batch failure `unknown`, including a step with no declared `recordingEffect`. +This applies to every caller, including a caller that supplies its own `invoke`. -For `responseFields`, `buildInteractionResponseData(...)` becomes the only construction site for interaction response payloads. -This prevents manual branches such as `fill @ref` from omitting `evidence`. -A small guard test follows repository precedent. -It fails if an interaction handler contains a manual `responseData = {` literal. +For `responseFields`, one `buildInteractionResponseData(...)` becomes the only construction site for interaction response payloads. +This deletes the class of bug where `fill @ref` rebuilt its response by hand and dropped `evidence`. +A small guard test, with precedent in this repository, fails if an interaction handler contains a hand-rolled `responseData = {` literal. ### Layer 3 — Prove: a contract suite generated from the registry -`test/integration/interaction-contract/` contains table-driven scenarios: fixture tree × command × forced path. -Permanent fixture trees preserve the real failure shapes: +`test/integration/interaction-contract/` holds table-driven scenarios: fixture tree × command × forced path. +The fixture trees are the real shapes that found this week's bugs, and the suite keeps them permanently: -- Closed drawer with all candidates offscreen → `offscreen_selector`/`offscreen_ref`. -- Stale or unknown `@ref` → `ref_not_found` with the stale-ref hint. -- Listed `@ref` or selector target without a usable center → `target_bounds_invalid`. - Bounds can be missing, non-finite, or negative. -- Listed `@ref` without a label for wait or scope → `ref_unlabeled`. -- Drawer item and visible twin → visible candidate wins despite on/offscreen ambiguity. -- Container with 0.07 px viewport overlap and offscreen center → refusal. +- Closed drawer (all candidates off-screen) → `offscreen_selector`/`offscreen_ref`. +- Stale or unknown `@ref` (errorTaxonomy) → `ref_not_found` beside the stale-ref hint. +- A listed `@ref` whose node has no usable centre (missing, non-finite, or negative bounds), or a selector target without one → `target_bounds_invalid`. +- A listed `@ref` with no label to wait on or to scope by → `ref_unlabeled`. +- Drawer item + visible twin (ambiguous on/off-screen) → the visible candidate wins. +- Edge-grazing container (0.07 px viewport overlap, center off-screen) → still refused. - Covered node → occlusion refusal. -- Non-hittable target with or without `--verify` → hint present or suppressed by evidence. -- Stripped-root tree without Application/Window → safe-default visibility. +- Non-hittable target ± `--verify` → hint present / suppressed by evidence. +- Stripped-root tree (no Application/Window) → safe-default visibility. + +Path forcing is explicit and test-only: `AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`. +Thus, cases stay stable when path-selection heuristics change. -Tests force paths explicitly with `AGENT_DEVICE_FORCE_INTERACTION_PATH=runtime|direct|native-ref`. -This test-only control keeps cases stable when path selection heuristics change. -The gate traverses the Layer-1 registry. -It fails if a non-waived cell has no tagged contract case. -Thus, construction enforces matrix coverage without reviewer memory. +The gate closes the loop. +It walks the Layer-1 registry and fails if any non-waived cell has no contract case tagged for it. +Coverage of the matrix therefore comes from construction, not from reviewer memory. ### Closing the gaps: a hybrid strategy, not one answer -Different guarantees require different mechanisms to close acknowledged gaps: +The acknowledged gaps close by different mechanisms, which depend on what the guarantee needs: -- **Runner-side parity for low-cost geometry-local rules.** Direct iOS offscreen/tappable-frame rules use pure geometry proven by golden tables. - They preserve the fast path's latency. -- **Delegation-on-error for semantic and rich-runtime cases.** These include `ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, and non-hittable refusal. - The fast path fails at low cost. - The runtime path supplies disambiguation and full diagnostics only when required. - **Delegation-on-error does not prove success-path parity.** - XCTest may find one hittable candidate that runtime rules would refuse or rank differently. - Those cells retain `gap:` waivers until parity tables or contract scenarios also prove success-path behavior. -- **Shared runtime preflight for native-ref.** The daemon snapshot already contains the ref's node. - Before `tapTarget`/`fillTarget`, examine offscreen, occlusion, and non-hittable conditions against that node. - A backend fast path can silently succeed, so delegation-on-error would never run. +- **Runner-side parity for cheap geometry-local rules** (offscreen/tappable-frame on the direct iOS path). + These rules are pure frame math, golden tables can prove them, and they keep the fast path fast. +- **Delegation-on-error for semantic and rich-runtime cases** (`ELEMENT_OFFSCREEN`, `AMBIGUOUS_MATCH`, `ELEMENT_NOT_FOUND`, non-hittable refusal). + The fast path fails cheaply. + The runtime path supplies disambiguation and full diagnostics only when they are necessary. + **Delegation-on-error is not success-path parity.** + It cannot catch the case where XCTest finds one hittable candidate that runtime rules would refuse or rank differently. + Those cells stay `gap:` waivers until parity tables or contract scenarios prove the success path too. +- **A shared runtime preflight for the native-ref path.** The ref came from a daemon snapshot, so the node is already available. + Check offscreen, occlusion, and non-hittable against that node *before* the call to `tapTarget`/`fillTarget`. + A backend fast path can silently "succeed", so delegation-on-error would never start there. ### Timeout policy joins the descriptor registry -Two manually maintained client lists caused the `wait` timeout defect: - -- `isExplicitTimeoutCommand` controlled request-envelope budgets. -- `shouldResetDaemonAfterRequestTimeout` controlled timeout daemon policy. - -Command descriptors in ADR 0008 replace both lists with declarations: +The `wait` timeout bug existed because two hand-maintained lists in the client held request-envelope budgets and on-timeout daemon policy. +The two lists were `isExplicitTimeoutCommand` and `shouldResetDaemonAfterRequestTimeout`. +Declarations on the command descriptors (ADR 0008 registry) replace both lists: ```ts timeoutPolicy: { @@ -284,10 +287,10 @@ timeoutPolicy: { } ``` -A completeness gate covers every public command. +A completeness gate covers all public commands. Read-only polling commands declare `preserve-daemon`. -The client derives the envelope from the declared budget source instead of command-name exceptions. -Capture-resolving interactions also declare `preserve-daemon` (#1105): +The client derives the envelope from the declared budget source, instead of special cases for command names. +Since #1105, the capture-resolving interaction commands also declare `preserve-daemon`: - `click`. - `fill`. @@ -297,141 +300,155 @@ Capture-resolving interactions also declare `preserve-daemon` (#1105): - `get`. - `is`. -Like `snapshot`, their dominant timeout cause is blocked accessibility capture. -A daemon reset would destroy every healthy app session it owns. +Their dominant timeout mode is the same blocked accessibility capture as `snapshot`. +A daemon reset there destroyed every healthy app session that the daemon owned. ## Consequences -- A new path or guarantee requires a whole-matrix decision. - tsc rejects unclassified cells, and the gate rejects untested cells. - Silent loss is structurally impossible. - Explicit waivers remain possible, visible, and linked to issues. -- Fast paths retain latency benefits with documented differences and costs. -- Golden tables become the single source of truth for cross-language rules. - The fake runner no longer provides a second implementation that can diverge. -- Initial classification includes `waived('gap: ...')` cells. +- A new dispatch path or a new guarantee forces a whole-matrix decision. + tsc does not compile an unclassified cell, and the gate does not pass an untested cell. + Silent erosion is structurally impossible. + Explicit waivers remain possible, but they are visible and can link to issues. +- Fast paths keep their latency wins. + Their divergence is priced and documented, instead of discovered on-device. +- The Swift/TypeScript split stops being a parity blind spot. + Golden tables are the single source of truth for cross-language rules. + The fake runner stops being a second implementation that drifts. +- The initial cost is mostly classification honesty. + The first registry contains `waived('gap: ...')` cells, and that is the purpose. For example, the maestro non-hittable fallback intentionally waives `offscreen`. - The debt becomes a reviewable list instead of undocumented knowledge. -- The registry also supplies documentation input for help/skill output, such as the guarantees of press. - This matters for small-model agents that read contracts without code. + The debt becomes a diffable list instead of folklore. +- The registry is also documentation input for help/skill output ("what press guarantees"). + This matters for small-model agents that only read the contract, never the code. ### 2026-08-07 amendment: mutating ambiguity fails fast -Element-14 demonstrated the owned success-path gap with four exact `Team Standup` matches in distinct accessibility subtrees. -Ordinary runtime selector resolution silently selected a different semantic target through visible/depth/area ranks. -Mutating selectors no longer use geometry to select between distinct subtrees. +Element-14 made the matrix's previously owned success-path gap real. +Four exact `Team Standup` matches in distinct accessibility subtrees reached ordinary runtime selector resolution. +Its visible/depth/area ranking silently chose a different semantic target. +Mutating selectors no longer use geometry to choose among distinct subtrees. The replacement contract is structural. -Multiple matches collapse only if all form one ancestor–descendant chain and every member resolves to the same actionable node. -Otherwise, mutation fails with `AMBIGUOUS_MATCH`. -It returns bounded snapshot candidate lines and a partial ref frame generation for immediate retry with one listed candidate. -On direct XCTest paths, this applies only to mutating dispatches. -They count all raw exact matches before hittability can select a candidate. -Multiple matches delegate to the runtime classifier. -Reads through `querySelector`, and thus the original `get`/`is`/`wait` paths, retain the prior rule: +Multiple matches collapse only if all matches form one ancestor–descendant chain and every member resolves to the same actionable node. +Otherwise, the mutation fails with `AMBIGUOUS_MATCH`, a bounded list of snapshot candidate lines, and a partial ref frame generation. +Thus, the caller can immediately retry one listed candidate. +On the direct XCTest path, this applies only to mutating dispatches. +They count all raw exact matches before hittability can select a winner, and they delegate multiple matches to the runtime classifier. +Reads (`querySelector`, and so `get`/`is`/`wait`) keep the prior rule: - Prefer the single hittable match. - Report ambiguity only if hittable matches compete. -Reads have no side effect to protect. -A closed failure would turn a decorative duplicate into an error. -The amendment below updates which paths still use `querySelector`. +The reason: a read has no side effect to guard, and a fail-closed read would turn a decorative duplicate into an error. Maestro's explicit expected-point/non-hittable compatibility path remains intentionally separate. ### 2026-09-11 amendment: one collapse for one control, at both doors -Two earlier clauses became stale after read paths changed. -The structural rule also needed to apply to reads: - -- `querySelector` no longer serves `get`/`is`/`wait`. - These reads resolve a capture through `readUnique`/`readAny`/`readText` in `packages/selectors/src/selector-pipeline.ts`. - The runner's hittable-preference rule therefore governs only remaining direct XCTest queries: direct-iOS touch fast path and offscreen target probe. -- A control's accessibility wrapper can make one selector match twice. - Regular iOS snapshots omit unverified hittability, so the wrapper-to-control relation cannot use that evidence. - A mutation-only collapse (#2482) let `press` succeed while `is visible` and `get attrs` reported ambiguity on the same screen. - `resolveUnverifiedWrapperControl` now places collapse beside its classification. - Uniqueness rows that otherwise refuse resolve the control: +Two clauses of the amendment above became stale when the read paths moved. +Also, the structural rule never reached the read door. + +- `querySelector` no longer backs `get`/`is`/`wait`. + Those reads resolve against a capture, through the `readUnique`/`readAny`/`readText` rows of `packages/selectors/src/selector-pipeline.ts`. + Thus, the runner's hittable-preference rule governs only the direct XCTest paths that still query it: the direct-iOS touch fast path and the off-screen target probe. +- A control reported through its own accessibility wrapper answers a selector twice. + A regular iOS snapshot omits unverified hittability, so the ladder that relates a wrapper to its control cannot fire. + #2482 collapsed that chain for mutating resolution only. + As a result, `is visible` and `get attrs` refused the same screen as ambiguous, while `press` tapped it. + The collapse now sits beside the classification that asks for it (`resolveUnverifiedWrapperControl`). + The collapse applies where a refusal was the answer. + The uniqueness rows resolve the control instead of reporting no match: - `is `. - `get attrs`. - `screenshot --crop-on`. - Rows that resolve before refusal remain unchanged. - A wrapper chain always resolves before refusal on those rows. - Depth makes `get text` rank the control above its wrapper. - `wait`/`is exists` use the first document-order match without requiring unique identity. - Candidate sets outside the one-control rule still refuse. - Examples include a cell with its internal button and matches in distinct subtrees. -- Replay verifies recorded targets by resolving their selectors under the same row's refusal rules. - Without equivalent collapse, the first replay could report identity mismatch for a target dispatch had resolved. - Verification therefore names the collapsed control, matching both dispatch and recorded identity. + The collapse does not touch a row that resolves before any refusal. + A wrapper chain always resolves before a refusal, because depth separates a wrapper from its control. + Thus, `get text` ranks onto the control. + `wait`/`is exists` answer from the document-order head, and do not ask which element was meant. + A candidate set that the rule does not recognize as one control still refuses. + A cell and the button inside it, or matches in distinct subtrees, are such candidate sets. +- Replay verifies a recorded target: it resolves the recorded selector again under the same row's refusal rules. + Thus, the screen that dispatch had resolved read as an identity mismatch on the first replay of the step. + Verification names the collapsed control too. + That control is the node that dispatch acted on, and the node that the recorded identity carries. ### Synthesized iOS gesture policy -Synthesized iOS gestures intentionally remain outside the element dispatch-path matrix: +The element dispatch-path matrix above intentionally does not include synthesized iOS gestures: - `scroll`. - Synthesized coordinate `tap`. - Synthesized `drag`. - Synthesized `sequence` tap/drag steps. -They resolve neither selectors nor refs. -Element-targeting guarantee claims would therefore mislead. -`RunnerTests+SynthesizedGesturePolicy.swift` retains runner-local rules for AX health, keyboard probes, frame sources, activation preflight, and XCTest-coordinate fallback. -This Swift policy is authoritative because the current table has only three behaviors: +They do not resolve selectors or refs, so a claim of the element-targeting guarantees would be misleading. +Their AX-health, keyboard-probe, frame-source, activation-preflight, and XCTest-coordinate fallback rules stay runner-local in `RunnerTests+SynthesizedGesturePolicy.swift`. +That Swift policy is the source of truth, because the current table has only three behaviors: -- Synthesized coordinate taps never probe keyboards and may use coordinate fallback. - This includes standalone taps, `sequence` steps, and the in-app `back` top-leading tap. -- Default iOS scroll probes keyboards only after known AX health. +- A coordinate synthesized tap never probes keyboards and may use the coordinate fallback. + This applies to a standalone tap, a `sequence` step, and the in-app `back` top-leading tap. +- Default iOS scroll probes keyboards only after AX is known healthy. It must not fall back to `XCUICoordinate`. -- Synthesized one-contact `gesture` plans may use coordinate fallback before known AX health. - After a snapshot reports unavailable AX, they stop using it. +- Synthesized one-contact `gesture` plans may still use the coordinate fallback before AX health is known. + After a snapshot stamps AX unavailable, they stop using it. -Gated XCTest policy tests cover the less obvious cases instead of a cross-language duplicate. -A sibling registry should be introduced only under either condition: +Gated XCTest policy tests cover the non-obvious parts, instead of a cross-language mirror. +A future sibling registry should be introduced only once one of these conditions is true: - Gesture policy grows beyond this small runner-local table. -- Host-side tools need to consume it directly. +- Host-side tooling needs to consume gesture policy directly. Synthesized coordinate contexts use screenshot dimensions as their only frame source. -They intentionally exclude cheaper sources such as: +They intentionally exclude cheaper frame sources, such as: - `app.frame`. - Windows. - Accessibility frames. - Native screen bounds. -Those sources can differ from full-screen screenshot coordinates on affected simulators. +The reason: those sources can diverge from full-screen screenshot coordinates on affected simulators. -Target-authored `gesture drag` uses the `target-drag` path because it resolves two elements. -It inherits element guarantees and ADR 0012 identity requirements for both endpoints. +Target-authored `gesture drag` is classified as the `target-drag` path above, because it resolves two elements. +Therefore, it inherits the element guarantees and the ADR 0012 identity requirements for both elements. Coordinate-authored drag remains outside this matrix. -[ADR 0013](0013-unified-gesture-plans.md) owns two-contact pan/pinch/rotate/transform plans. -That contract remains separate from this element-targeting matrix. -A coordinate gesture does not acquire selector/ref guarantees because its native executor uses XCTest. +The typed gesture-plan contract in [ADR 0013](0013-unified-gesture-plans.md) now owns two-contact pan/pinch/rotate/transform planning. +That contract remains a sibling of this element-targeting matrix. +A coordinate gesture does not acquire selector/ref guarantees only because its native executor uses XCTest. + +## Migration plan + +Each step lands green and is independently useful. +Steps 1, 2, 4, and 5 are complete, and git history keeps them. +Step 3 is not complete: + +3. **Golden fixture tables** + `TapPointPolicy` Swift extraction + the three consumers (vitest, XCTest, fake runner). ## Alternatives considered -- **Delete fast paths and use one resolution path.** Unacceptable. - Direct iOS avoids a full snapshot round trip per interaction. - The ref fast path is central to replay throughput. - Their implicit compromises require explicit contracts. -- **Use typestate/branded proof types.** Proof that response guards ran cannot cross the wire into Swift. - Proof tokens would spread through every signature for little additional benefit over the registry and gate. - Types enforce declaration completeness, while tests prove declared behavior. -- **Add integration tests without a registry.** This retains the existing failure mode with more effort. - Without a coded matrix, no mechanism requires a new path to acquire the existing suite. +- **Delete the fast paths** (single resolution spine): unacceptable. + The direct iOS path saves a full snapshot round trip per interaction. + The ref fast path is the backbone of replay throughput. + The problem is not that fast paths exist. + The problem is that their trade-offs were implicit. +- **Typestate/branded types** ("cannot construct a response without proof the guards ran"): this cannot reach across the wire into Swift. + Also, the proof tokens would spread through every signature, for a marginal gain over the registry-plus-gate split. + Types are used where they are strongest: completeness of the declaration. + Tests are used where they are strongest: truth of the declaration. +- **More integration tests without the registry**: this is the status quo plus effort. + Without the matrix as code, nothing forces a new path to acquire the existing suite. + That is exactly how this week's bugs happened. ### Optional observation before an iOS coordinate tap A coordinate tap must not depend on a preceding XCTest snapshot failure. -Its optional text-input lookup may establish concrete identity for a later bare `type`. +Its optional text-input lookup may establish a concrete identity for a later bare `type`. An absent or unavailable lookup establishes no typing witness. -A runner snapshot penalty can skip this work, but only as a performance optimization. +A runner snapshot penalty can skip this work, but the penalty is only a performance optimization. The lookup owns a thread-bound issue scope in the runner recorder and returns a typed result. -Any recorded issue discards partial candidates, including an issue otherwise handled by AX suppression. +Any recorded issue discards partial candidates, including an issue that AX suppression otherwise handles. The scope excludes gesture dispatch and required text-entry reads. -Those failures retain existing mutation-outcome rules. +Those failures retain the existing mutation-outcome rules. The iOS PR lane exercises: @@ -441,6 +458,7 @@ The iOS PR lane exercises: - Failures outside the optional observation scope. These tests must not seed a snapshot penalty to make the first tap safe. -Before the recorder forwards optional-read issues to XCTest, it consumes them. + +The recorder consumes optional-read issues before it forwards issues to XCTest. XCTest's expected-failure API must not own this scope. -In a long-lived command test, it can complete the enclosing test even if the command response succeeds. +The reason: in a long-lived command test, that API can complete the enclosing test even if the command response succeeds. From 65d01bbf929ce1ad098871aa1beb5858302e3fb4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 42/55] docs(adr): restore the frozen meaning in ADR 0013 Restore the Android rule that gestures do not stop or restart the snapshot session (#1275), which the language commit dropped. Restore the codec amendment date (#1216), "only remaining callers", and "nothing left to migrate off". Make the glance rules match the body: removed-argument rejection, viewport failure code, 60 s scope, and drag resolve-versus-replay-verify. Restore dropped conditions, owners, and causal links in the Android, Apple, codec, and drag sections. --- docs/adr/0013-unified-gesture-plans.md | 287 +++++++++++++------------ 1 file changed, 152 insertions(+), 135 deletions(-) diff --git a/docs/adr/0013-unified-gesture-plans.md b/docs/adr/0013-unified-gesture-plans.md index 500dc544c9..1566af8feb 100644 --- a/docs/adr/0013-unified-gesture-plans.md +++ b/docs/adr/0013-unified-gesture-plans.md @@ -10,14 +10,18 @@ Completed process history is in git history. - Normalize public gesture input once in `packages/contracts/src/gesture-normalization.ts`. - Plan canonical intent in `packages/contracts/src/gesture-plan.ts`. Keep contact topology separate from motion. -- Reject removed timed public forms with actionable `INVALID_ARGS` messages. -- Keep Maestro's `endpoint-hold` profile private. Public `pan` uses `timed-pan`. -- Every injected sample must fit the freshly resolved active-app interaction viewport. Never clamp or distort the plan. +- Reject removed deprecated arguments with actionable `INVALID_ARGS` messages. Do not silently reinterpret them. +- Carry Maestro's `endpoint-hold` profile only on the private daemon wire. A genuine public `pan` uses `timed-pan`. +- Every injected sample must fit the freshly resolved active-app interaction viewport. + Otherwise, the request fails before injection with `GESTURE_TRAJECTORY_OUT_OF_BOUNDS`. + The planner does not clamp points, cache the viewport, or distort the requested components. - Android touch execution never falls back to `adb input swipe`. -- Keep the positional `.ad` codec and shared `PUBLIC_GESTURE_SYNTAX` arity table. -- Repeated swipe and pause schedules must fit within 60 seconds. -- Before target-authored drag, resolve and verify both endpoints. Admit it only on Android touch devices and iOS/iPadOS. -- iOS timing changes require independent app-observed live simulator evidence. +- On Android, gestures do not stop or restart the snapshot session. +- Keep the positional `.ad` codec. One `PUBLIC_GESTURE_SYNTAX` table defines argument arity for the CLI argv parse and the `.ad` line parse. +- For repeated coordinate swipes, the combined planned gesture and pause schedule must fit within 60 seconds. +- For target-authored drag, resolve both endpoints before device injection. On replay, verify both identities before pointer-down. +- Admit target-authored drag only on Android touch devices and iOS/iPadOS. +- iOS timing changes require live simulator evidence from an independent app-observed postcondition. ## Context @@ -25,40 +29,40 @@ Several private boundaries previously interpreted gesture intent: - Command aliases produced positional strings. - Daemon dispatch parsed them again. -- `Interactor` exposed parallel semantic methods. +- The `Interactor` exposed parallel semantic methods. - Each platform derived its own two-contact geometry. -This duplicated validation, responses, and behavior behind CLI, Node.js, and MCP. +This duplicated validation, responses, and behavior behind the three public surfaces: CLI, Node.js, and MCP. Two-contact geometry also differed by platform. -Android used a fixed radius, while Apple derived radius from the app frame. +Android used a fixed radius, while Apple derived a radius from the app frame. Neither path validated every planned point against the active interaction viewport before injection. -`scroll` remains separate because it owns viewport/edge traversal and content-state verification, not one physical gesture. -Its Android physical movement still uses the Android planned-touch executor. +`scroll` remains a separate command because it owns viewport/edge traversal and content-state verification, not a single physical gesture. +Its Android physical movement still lowers to the Android planned-touch executor. ## Decision Public gesture inputs normalize once in `packages/contracts/src/gesture-normalization.ts`. This is the explicit public compatibility boundary. -It produces canonical semantic intent before runtime entry. -Removed deprecated arguments receive actionable `INVALID_ARGS` errors instead of silent reinterpretation: +Canonical semantic intent is produced before the input enters the runtime. +These removed deprecated arguments are rejected with actionable `INVALID_ARGS` messages, not silently reinterpreted: - Timed `swipe`. - Timed `gesture fling`. - `gesture rotate` `velocity`. -The private daemon wire may still carry compatibility-only hints absent from public surfaces. -Maestro timed swipes use `internal.gestureExecutionProfile: 'endpoint-hold'` with canonical `pan` input. -This preserves iOS fast-swipe-then-hold behavior. +The private daemon wire remains free to carry compatibility-only hints that do not appear on public surfaces. +Maestro timed swipes use `internal.gestureExecutionProfile: 'endpoint-hold'` and still route through the canonical `pan` input. +This preserves the iOS fast-swipe-then-hold behavior. The runtime plans canonical intent in `packages/contracts/src/gesture-plan.ts`. Contact topology is separate from motion: -- One contact: pan, fling, or target-authored drag with a complete pointer trajectory and explicit execution profile. -- Two contacts: pan, pinch, rotate, or transform with two complete synchronized trajectories. +- One contact: pan, fling, or target-authored drag with a complete pointer trajectory and an explicit execution profile. +- Two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories. -Public `swipe` without duration remains an alias for fixed-duration fling. +Public `swipe` without a duration remains an alias for a fixed-duration fling. These timed public forms are rejected: - `swipe x1 y1 x2 y2 durationMs`. @@ -67,192 +71,205 @@ These timed public forms are rejected: - `gesture rotate degrees x y velocity`. For deliberate timed translation, callers must use `gesture pan`. -For rotation, they must use `gesture rotate` without `velocity`. -Maestro swipes normalize to canonical `pan` with origin, delta, and durationMs. -They carry `endpoint-hold` through daemon-internal `internal.gestureExecutionProfile`. -This matches Maestro's XCTest driver without a public execution-profile option. -If duration is absent, Maestro supplies its 400 ms default. -A public `pan` uses `timed-pan`. -Thus, compatibility paths retain release behavior without a new semantic intent. +For rotation, callers must use `gesture rotate` without `velocity`. +Maestro-authored swipes normalize to the canonical `pan` input (origin, delta, durationMs). +They carry the `endpoint-hold` execution profile through a daemon-internal compatibility seam (`internal.gestureExecutionProfile`). +This preserves the iOS fast-swipe-then-hold behavior that matches Maestro's XCTest driver. +The public command surface does not expose the profile. +If the duration is omitted, Maestro materializes its 400 ms default. +A genuine public `pan` uses the `timed-pan` profile. +Thus, compatibility-only paths retain their release behavior and do not become a new semantic intent. -The plan retains intent even when aliases share an executor: +Each intent fixes some plan components: - Pinch fixes translation and rotation at zero. - Rotate fixes translation at zero and scale at one. - Two-finger pan fixes scale at one and rotation at zero. - Transform can apply all three components atomically. +Intent remains on the plan even when aliases share an executor. + The planner owns deterministic multi-touch geometry. -Contacts start at -90 degrees, except Android pinch starts horizontally. -Common vertical app scroll containers capture vertical Android pinches before the pinch recognizer activates. -An explicit planning profile also preserves dense two-contact frame-count conventions. +Contacts start at -90 degrees, except that Android pinch starts horizontally. +The reason is that common vertical app scroll containers capture a vertical pinch before the pinch recognizer activates. +The same explicit planning profile preserves the proven frame-count convention for dense two-contact trajectories. Android rounds the duration/16 ms frame count. Apple truncates it. -The Android transport lowerer uses the same Android sampling profile for one-contact endpoint plans. -These profiles are planner inputs, not trajectories generated by adapters. +The Android transport lowerer uses that same Android sampling profile for one-contact endpoint plans. +These are planner inputs, not adapter-generated trajectories. -For pinch, the larger initial/final span is 40% of the viewport's shorter side, which preserves proven Apple geometry. -Other two-contact intents use 25% for compact translation and rotation trajectories. -Requested scale determines the other span. +For pinch, the larger of the initial and final spans is 40% of the viewport's shorter side. +This preserves the proven Apple pinch geometry. +Other two-contact intents use 25% to keep translation and rotation trajectories compact. +The requested scale determines the other span. Both spans must satisfy a 48-point reliability floor. -Combined transforms progress translation, scale, and rotation together in one uninterrupted two-contact sequence. -Recognizers observe every intent without adapter-generated geometry. -The planner does not clamp points, cache the viewport, or distort requested components. +Combined transforms progress translation, scale, and rotation together inside one uninterrupted two-contact sequence. +Thus, recognizers observe every intent, and no adapter regenerates geometry. +The planner does not clamp points, cache the viewport, or distort the requested components. Every injected sample must fit the freshly resolved active-app interaction viewport. Otherwise, the request fails before injection with `GESTURE_TRAJECTORY_OUT_OF_BOUNDS` and actionable details. -Span and angle remain internal because no established automation use case justifies public controls. +Span and angle remain internal because no established automation use case justifies a public tuning surface. Platform adapters consume the canonical plan: -- **Android.** `executeAndroidTouchPlan` receives gesture plans and physical movement for scroll and long-press. - If provider-native touch injection is available, it uses that provider. - Otherwise, it uses the bundled instrumentation helper. +- **Android.** The `executeAndroidTouchPlan` adapter seam sends planned touch. + Planned touch includes gesture plans and the physical movement for scroll and long-press. + If provider-native touch injection is available, the seam sends planned touch to it. + Otherwise, the seam sends planned touch to the bundled instrumentation helper. `packages/platform-android/src/touch-plan-lowering.ts` lowers one-contact endpoint plans to approximately 16 ms transport samples before either injection path. Controlled directional scrolls accelerate for one frame, then decelerate through release within the requested duration. - They append no extra movement. + They do not append a tail. Inertial scrolls and general one-contact plans retain linear interpolation. Two-contact plans retain their exact planned samples. Easing reduces release momentum but does not guarantee an exact content offset. - Transport types require samples strictly denser than the canonical endpoint pair. - Thus, omitted lowering causes a type error at injection boundaries instead of silently sparse input. + Transport samples are typed as strictly denser than the canonical endpoint pair. + Thus, a skipped lowering is a type error at the injection seams, not a silently sparse gesture. A stationary long-press needs no viewport on the helper path. - Only provider-native touch adds the paired provider-owned viewport. + The executor adds the paired provider-owned viewport only for provider-native touch. Android touch execution never falls back to `adb input swipe`. - Public scroll durations below one 16 ms planner frame normalize to that minimum and report the executed duration. - Scroll evidence reports absolute injected coordinates against zero-origin extents that include viewport offset. + Public scroll durations below one 16 ms planner frame normalize to that physical minimum and report the executed duration. + Scroll evidence reports absolute injected coordinates against zero-origin extents that include the viewport offset. Android permits only one instrumentation owner of `UiAutomation`. Snapshot capture, gesture viewport resolution, and planned-touch injection therefore share one bundled automation helper. A live persistent helper session executes touch commands directly. - Without one, the same helper runs one-shot. + Without a live session, the same helper runs one-shot. + Gestures do not stop or restart the snapshot session (amended 2026-07, issue #1275). - **Apple.** iOS lowers one-contact endpoint-hold plans to the established fast-swipe synthesis profile. - It reaches the endpoint in 100 ms, holds for the planned duration, then lifts, as Maestro's XCTest driver does. - One-contact plans are linear and carry only start/end samples with the authored duration between them. + That profile reaches the endpoint in 100 ms, then holds there for the planned duration before it lifts. + This matches Maestro's XCTest driver. + One-contact plans are linear and therefore carry only their start and end samples, with the authored duration between them. Two-contact plans convert every planned point to native orientation and send the exact arrays to the private XCTest event bridge. - macOS lowers one-contact plans to its drag executor. - tvOS lowers them to remote direction. - Core admission and the Apple adapter consume the same shared multi-touch support policy. + macOS lowers a one-contact plan to its drag executor. + tvOS lowers a one-contact plan to remote direction. + Core admission and the Apple adapter both consume the same shared multi-touch support policy. Multi-touch remains capability-gated to iOS simulators. - iOS intentionally retains three one-contact motion schedules behind the shared private-XCTest event bridge. - Apple's public drag API independently models [initial hold, movement velocity, and destination hold](https://developer.apple.com/documentation/xcuiautomation/xcuicoordinate/press%28forduration%3Athendragto%3Awithvelocity%3Athenholdforduration%3A%29). - UIKit reports finger-up velocity with the predicted [resting content offset](https://developer.apple.com/documentation/uikit/uiscrollviewdelegate/scrollviewwillenddragging%28_%3Awithvelocity%3Atargetcontentoffset%3A%29). - It then applies the configured [post-lift deceleration rate](https://developer.apple.com/documentation/uikit/uiscrollview/decelerationrate-swift.property). - Movement distribution over time is therefore observable gesture input, not merely adapter implementation. + iOS deliberately retains three one-contact motion schedules behind the shared private-XCTest event bridge. + Apple's public drag API models the phases independently as an [initial hold, movement velocity, and destination hold](https://developer.apple.com/documentation/xcuiautomation/xcuicoordinate/press%28forduration%3Athendragto%3Awithvelocity%3Athenholdforduration%3A%29). + UIKit also reports the finger-up velocity together with the scroll view's predicted [resting content offset](https://developer.apple.com/documentation/uikit/uiscrollviewdelegate/scrollviewwillenddragging%28_%3Awithvelocity%3Atargetcontentoffset%3A%29). + UIKit then applies the scroll view's configured [post-lift deceleration rate](https://developer.apple.com/documentation/uikit/uiscrollview/decelerationrate-swift.property). + The distribution of movement over time is therefore observable gesture input, not an adapter implementation detail. - The schedules produce different velocity profiles: + The three schedules shape that velocity differently: - - `endpoint-hold` moves for 100 ms, then remains stationary before lift. - - `timed-pan` and target-authored drag submit authored samples unchanged. - They preserve piecewise-linear movement and explicit source/destination holds. - - Fused runner `scroll` reuses `endpoint-hold` for inertial release. - Controlled release expands movement into evenly spaced samples, approximately one per 16 ms, capped at 30. - It uses cubic ease-out `s(t) = 1 - (1 - t)³`. + - `endpoint-hold` moves quickly for 100 ms and becomes stationary before lift. + - `timed-pan` and target-authored drag submit the authored samples unchanged. + They preserve piecewise-linear movement plus explicit source and destination holds. + - The runner's fused `scroll` reuses `endpoint-hold` for inertial release. + For controlled release, the fused `scroll` instead expands the movement to evenly spaced samples, one per approximately 16 ms, capped at 30. + Controlled release uses cubic ease-out `s(t) = 1 - (1 - t)³`. - A linear segment retains constant velocity through its endpoint unless a destination hold follows. - Cubic ease-out decelerates from first movement and has zero slope at lift. + A linear segment has constant movement velocity through its endpoint, unless a destination hold follows it. + Cubic ease-out decelerates from the first movement and has zero slope at lift. Its starting velocity is three times its average. - Thus, identical endpoints and total durations can produce different recognition and deceleration outcomes. - Neither destination hold nor analytically zero endpoint slope alone proves controlled release. - XCTest event sampling and app recognizer thresholds can still produce post-lift motion. - Live evidence must therefore measure content offset after pointer-up. - - Live iOS characterization in [issue #1586](https://github.com/callstack/agent-device/issues/1586) compared an eased sampled schedule with linear schedules. - In the tested range, they crossed the same fling-recognizer thresholds. - However, post-release ScrollView positions and long-duration recognition differed materially. - This distinction is intentional Apple adapter policy, not a second interpretation of `GesturePlan`. - Changes require live evidence for recognizer activation and post-release content movement. -- **WebDriver.** Supported plans become synchronized W3C pointer action sources. - One-contact endpoint plans produce pointer down, one timed W3C `pointerMove` from start to end, and pointer up. + Thus, identical endpoints and total durations can produce different recognizer and deceleration outcomes. + Neither a destination hold nor an analytically zero endpoint slope proves a controlled release by itself. + XCTest event sampling and app recognizer thresholds can still leave observable post-lift motion. + Live evidence must therefore measure the resulting content offset after pointer-up. + + Live iOS characterization in [issue #1586](https://github.com/callstack/agent-device/issues/1586) compared an eased sampled schedule with the linear schedules. + It confirmed that distinction. + In the tested range, the schedules crossed the same fling-recognizer thresholds. + However, they produced materially different post-release ScrollView positions and long-duration recognition behavior. + This distinction is intentional policy at the Apple adapter boundary, not a second interpretation of a `GesturePlan`. + Changes require live evidence for both recognizer activation and post-release content movement. +- **WebDriver.** WebDriver lowers a supported plan to synchronized W3C pointer action sources. + A one-contact endpoint plan becomes pointer down, one timed W3C `pointerMove` from start to end, and pointer up. The driver owns interpolation across that W3C tick. Multi-touch remains capability-gated until a provider proves it. -`Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of one method per semantic alias. -The old scalar Apple/Android multi-touch executors are deleted. -The public alias-to-positionals-to-reparse route is also deleted. -`.ad` retains its established positional syntax through one named codec. +The `Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of one method per semantic alias. +The old scalar Apple and Android multi-touch executors are deleted. +The public-command alias-to-positionals-to-reparse route is also deleted. +`.ad` keeps its established positional syntax through one named codec. CLI, Node.js, and MCP send structured input. -Providers should compose transport/device bindings with the shared platform adapter instead of duplicating the interaction runtime. +Providers should compose transport/device bindings with the shared platform adapter instead of a reimplementation of the interaction runtime. -The `.ad` codec owns script syntax. -It is not a compatibility shim and is not scheduled for removal. -Its callers are CLI argv parse and `.ad` line parse, both current public syntax. -A structured `.ad` gesture payload was rejected because it would reduce readability and text search without behavioral benefit. +That `.ad` codec is the script format's own syntax, not a compatibility shim. +It is not scheduled for removal (amended 2026-07, issue #1216). +Its only remaining callers are the CLI argv parse and the `.ad` line parse. +Both are the current public syntax, so nothing is left to migrate off. +A structured `.ad` gesture payload was rejected. +It would make recordings unreadable and impossible to search with grep, for no behavioral gain. One table, `PUBLIC_GESTURE_SYNTAX`, defines argument arity for both callers. -Thus, removal of a CLI form removes the `.ad` form in the same edit. -If a script retains a removed positional, parsing fails before any replay device action. +Thus, a form removed from the CLI is removed from `.ad` in the same edit. +If a `.ad` script still carries a removed positional, the script fails when it is parsed. +This failure occurs before the replay executes any device action. The error names the offending line and its rewrite. -Only arity is checked at parse time. +Arity is the only thing checked at parse time. `${VAR}` tokens resolve after planning, and interpolation never splits a token. -Thus, counts are known before values. -The public [gesture migration guide](https://oss.callstack.com/agent-device/docs/migrating-gestures) defines this removal process: +Thus, the count is decidable at parse time, but the values are not. +These inputs follow a removal process. +The public [gesture migration guide](https://oss.callstack.com/agent-device/docs/migrating-gestures) documents it: 1. Announce the removal. 2. Warn for one minor release. 3. Publish the migration guide. -4. Prove that the repository has no remaining use. +4. Prove that the repository is clean. 5. Remove the branch and its tests. -The public command contract and daemon trust boundary bound repeated coordinate swipes. +Repeated coordinate swipes are bounded at the public command contract and at the daemon trust boundary. Individual count and pause limits prevent pathological fields. The combined planned gesture and pause schedule must fit within 60 seconds. -Thus, individually valid fields cannot create an unbounded session lock. +Thus, valid fields cannot compose into an unbounded session lock. -Public two-finger pan is additive through `pointerCount?: 1 | 2` and CLI `--pointer-count 2`. +Public two-finger pan is additive: `pointerCount?: 1 | 2` on pan, and CLI `--pointer-count 2`. Omission remains one contact. -Responses share canonical `kind`, `durationMs`, `pointerCount`, `from`, and `to`, followed by backend evidence. -Recording/replay retains public command identity and session semantics. +Responses share the canonical `kind`, `durationMs`, `pointerCount`, `from`, and `to` fields, followed by backend evidence. +Recording/replay keeps its existing public command identity and session semantics. -Target-authored drag is additive through `gesture drag `. -Each endpoint accepts a selector or pinned snapshot ref. -Both endpoints resolve before injection. -Their center points become one uninterrupted single-pointer plan: +Target-authored drag is additive: `gesture drag `. +Each endpoint accepts a selector or a pinned snapshot ref. +Both endpoints resolve before device injection. +Their center points then lower to one uninterrupted single-pointer plan: 1. Hold at the source. 2. Move over the authored duration. -3. If requested, hold at the destination. +3. Optionally, hold at the destination. 4. Release. -The caller receives both endpoint-resolution disclosures and portable selector chains. -Recordings replace both session-local refs with those chains. -One `targets-v1` annotation carries independent source and destination identity evidence. +Both endpoint-resolution disclosures and portable selector chains are returned to the caller. +Recordings replace session-local refs at both endpoints with those selector chains. +Recordings also attach one `targets-v1` annotation that contains independent source and destination identity evidence. Before pointer-down, replay verifies both identities. -It guards each independent endpoint resolution against the verified element. -Ref admission occurs for both endpoints before either dispatch. +Replay also guards both independent endpoint resolutions against the verified elements. +Ref admission happens for both endpoints before either endpoint is dispatched. After the gesture, the usual mutation boundary expires the frame. -Only Android touch devices and iOS/iPadOS admit target-authored drag. -Their adapters preserve the complete source-hold, timed-movement, and destination-hold plan. +This contract includes the source hold, timed movement, and destination hold phases. +Thus, target-authored drag is admitted only on Android touch devices and iOS/iPadOS, whose adapters preserve the full plan. ADR 0011's coordinate path remains unchanged. -Coordinate-authored gestures resolve neither selectors nor refs and cannot claim element-targeting guarantees. -Before coordinate gesture execution, target-authored drag runs shared selector/ref preflight. -Tests prove endpoint resolution and recording contracts at that composition boundary. +Coordinate-authored gestures do not resolve selectors or refs and therefore cannot claim element-targeting guarantees. +Before it enters the coordinate gesture executor, target-authored drag explicitly runs the shared selector/ref resolution preflight. +Its endpoint resolution and recording contracts are tested at that composition seam. ## Consequences -- CLI, Node.js, MCP, runtime, and adapters share one normalization and planning model. -- A new ergonomic alias does not require a platform implementation. -- One-finger pan remains default, and explicit two-finger pan retains pan intent. -- Each gesture resolves its active viewport. +- CLI, Node.js, MCP, runtime, and platform adapters share one normalization and planning model. +- A new ergonomic gesture alias does not add a platform implementation. +- One-finger pan remains the default, and explicit two-finger pan retains pan intent. +- The active viewport is resolved for each gesture. Rotation, keyboard, and window changes therefore do not use stale geometry. -- On bare ADB, Android scroll and long-press require the bundled automation helper, the snapshot helper APK, and `UiAutomation`. - Helper install or runtime failure returns directly instead of approximate `adb input swipe` fallback. +- On bare ADB, Android scroll and long-press require the bundled automation helper, which is the snapshot helper APK, and `UiAutomation`. + Helper installation or runtime failure is surfaced directly. + It does not degrade to an approximate `adb input swipe`. - Canonical one-contact plans contain only two samples. A 10-second pan therefore has two shared-plan samples instead of 626. - Android expands them only at the transport boundary. - Two-contact plans remain cadence-bounded because synchronized geometry belongs to their contract. -- Unit tests cover plan shape, Android lowering, helper/provider payloads, and WebDriver actions. - They cannot prove timing or delivery inside the private XCTest bridge. + Android expands that plan only at the transport boundary. + Two-contact plans remain cadence-bounded because their synchronized geometry is part of their contract. +- Unit tests cover canonical plan shape, Android lowering, helper/provider payloads, and WebDriver action construction. + They cannot prove timing or event delivery inside the private XCTest bridge. iOS timing changes therefore require live simulator evidence from an independent app-observed postcondition. - Runner start/end uptime deltas can supplement it. - Request echoes or internal timing alone do not prove recognition of the intended gesture. + Runner start/end uptime deltas can supplement that evidence. + Request echoes or internal timing alone do not prove that the app recognized the intended gesture. ## Alternatives Considered -- **Keep positional aliases and share only geometry.** Rejected because validation, response, and routes would retain two implementations. -- **Make platform gesture APIs authoritative.** Rejected because their timing, geometry, and recognition differ and cannot define cross-platform semantics. -- **Make swipe a public synonym for pan.** Rejected because established gesture vocabulary distinguishes quick directional fling/swipe from deliberate timed pan. -- **Add `two-finger-pan`.** Rejected because pointer count defines topology, not new motion intent. -- **Expose span/angle controls.** Rejected until a concrete automation use case requires them. -- **Consolidate scroll semantics.** Rejected because edge/content verification is distinct. - Only Android physical touch execution is shared. +- **Keep positional aliases and share only geometry math.** Rejected because validation, response, and routing would still have two implementations. +- **Make platform gesture APIs the source of truth.** Rejected because their timing, geometry, and recognizer behavior differ and cannot provide cross-platform semantics. +- **Make swipe a public synonym for pan.** Rejected because established gesture vocabulary treats a fling/swipe as a quick directional throw and pan as deliberate timed translation. +- **Add `two-finger-pan`.** Rejected because pointer count is topology, not a new motion intent. +- **Expose span/angle controls.** Rejected until a concrete automation use case needs them. +- **Consolidate scroll command semantics.** Rejected because edge/content verification is distinct. + Only its Android physical touch execution is shared. From 4d5ac5ff44b68bcbaae027243964a9b35f4dccd2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 43/55] docs(adr): restore the frozen meaning in ADR 0014 Restore ADR 0014 meaning that the STE rewrite changed: "implemented and enforced on `main`", the staged-plan pointer that code comments use, and the PR #1241 status sentence. Restore dropped conditions and quantifiers ("only ... and only with their epoch", "any", "every", "all reach the generic daemon leaf", "on that platform"), the side-effect "seam" term, "not the transition site", and the base ADR-composition and Consequences claims. --- docs/adr/0014-session-ref-frame-lifetime.md | 601 ++++++++++---------- 1 file changed, 313 insertions(+), 288 deletions(-) diff --git a/docs/adr/0014-session-ref-frame-lifetime.md b/docs/adr/0014-session-ref-frame-lifetime.md index b37b0cb691..45d87bea33 100644 --- a/docs/adr/0014-session-ref-frame-lifetime.md +++ b/docs/adr/0014-session-ref-frame-lifetime.md @@ -2,64 +2,72 @@ ## Status -Accepted. Enforcement applies across all platforms. - -Completed process history is in git history. +Accepted. +This decision is implemented and enforced on `main` across all platforms. Fresh live evidence exercised every production seam except Android blocking-dialog recovery. -That seam has an accepted evidence gap because no deterministic procedure can raise the system dialog on demand. -Fixture regressions fix the expected transition and abort behavior in `android-system-dialog-ref-frame.test.ts` and `interaction-touch-android-readiness.test.ts`. +Android blocking-dialog recovery shipped as a documented, accepted gap, because no deterministic repro exists to raise the system dialog on demand. +Fixture regressions pin the transition and abort logic of that seam: `android-system-dialog-ref-frame.test.ts` and `interaction-touch-android-readiness.test.ts`. + +The staged migration plan is in the git history of this file. +PR #1241 was an earlier compatible transitional fix, not this architecture migration. ## Rules at a glance +This section is a normative summary. [Decision](#decision) contains the binding contracts and edge cases. -`CONTEXT.md` defines ref-frame vocabulary. - -- A session owns at most one **ref frame**, separate from its latest operational observation, `session.snapshot`. - The frame holds an epoch exposed as `refsGeneration`, immutable source tree, `active`/`expired` state, and `all` or bounded issuance scope. - It is one value on `SessionState`. - Transitions replace it whole and never edit it in place. - Nominal `#`-private fields prevent construction, edits, or derivation outside `packages/daemon/src/core/ref-frame.ts`. - The field-owner gate covers unchanged whole-frame moves, including field clearance and assignment from another session. +The ref-frame vocabulary is promoted into `CONTEXT.md`. + +- A session owns at most one **ref frame**. + The ref frame is the authorization namespace for mutation refs. + It is separate from the latest operational observation (`session.snapshot`). + The frame holds an epoch exposed as `refsGeneration`, an immutable source tree, an `active`/`expired` state, and an `all` or bounded issuance scope. + The frame is one value on `SessionState`. + A transition replaces the frame whole and never edits it in place. + The frame type is nominal (`#`-private fields). + Thus, no module outside `packages/daemon/src/core/ref-frame.ts` can construct a frame, edit a frame, or derive a frame from an existing frame. + The type cannot judge a whole frame that moves unchanged, for example when code clears the field or assigns the frame of another session. + The field-owner gate still covers these moves. - A complete snapshot activates an `all` frame. - `find`, settled diffs, and replay divergence screens activate bounded partial frames that supersede the prior frame. + `find`, settled diffs, and replay divergence screens activate a bounded partial frame that supersedes the prior frame. Internal read-only captures never activate, reindex, or expire a frame. -- Every mutating leaf expires the frame after pre-action guards, immediately before the device operation. - No success-only rollback occurs. - Post-dispatch failure leaves the frame expired. +- Every mutating leaf expires the frame at the side-effect seam: after all pre-action guards, immediately before the device operation. + There is no success-only rollback. + A post-dispatch failure still leaves the frame expired. - Mutation admission requires an active frame whose epoch and issuance scope authorize the ref. - Typed rejection reasons have this order: + Rejections carry typed reasons in this order: 1. `ref_frame_expired`. 2. `ref_generation_mismatch`. 3. `plain_ref_requires_complete_frame`. 4. `ref_not_issued`. -- Read-only refs remain fail-open with structured staleness warnings only while retained frame evidence can resolve them. - Missing evidence fails instead of resolving against a newer observation by positional coincidence. -- Every daemon command declares `RefFrameEffect` (`preserve` | `may-invalidate` | `delegated`) in its descriptor daemon facet. - A completeness gate enforces registry classification. +- Read-only ref consumers stay fail-open with a structured staleness warning only while the frame retains the evidence of the ref. + Missing evidence fails, instead of resolving by positional coincidence against a newer observation. +- Every daemon command declares a `RefFrameEffect` (`preserve` | `may-invalidate` | `delegated`) in its descriptor daemon facet. + A completeness gate covers the registry classification. ## Context Snapshot refs such as `@e12` are positional names inside one accessibility-tree namespace. -They are inexpensive and effective within that namespace. -They do not identify an element across navigation, layout changes, or captures with different positional order. -A generation pin such as `@e12~s42` identifies the issuing namespace. -It does not stabilize the element. +They are cheap and effective within that namespace. +But they do not identify an element across navigation, layout changes, or another capture whose positional order is different. +A generation pin such as `@e12~s42` identifies the namespace that issued the ref. +The pin does not make the element stable. -The session model combines two distinct concepts in `session.snapshot`: +The current session model mixes two different concepts in `session.snapshot`: -1. The latest operational observation for selector matches, verification, settled observations, Android freshness, overlays, and diagnostics. -2. The caller-authorized tree whose refs may authorize later mutations. +1. The latest operational observation that selector matching, verification, settling, Android freshness, overlays, and diagnostics use. +2. The caller-authorized tree whose refs a later mutation may use. -This combination causes failures in both directions. -A mutation can change the device without another capture, so old positional refs appear current. -An internal read-only capture can replace `session.snapshot` and invalidate refs despite no caller mutation. -Partial `find` results, settled diffs, and replay divergence screens can clear the coarse stale marker for a whole tree after exposing few refs. +This mix causes failures in both directions. +A mutation can change the device without another tree capture, so the old positional refs appear current. +Conversely, an internal read-only capture can replace `session.snapshot` and invalidate refs, even though the caller performed no mutation. +Partial outputs, such as `find`, settled diffs, and replay divergence screens, can then clear the coarse stale marker for the whole replacement tree. +They can do this even though they exposed only a few refs. Issue [#1239](https://github.com/callstack/agent-device/issues/1239) and PR [#1241](https://github.com/callstack/agent-device/pull/1241) demonstrate the first failure on iOS. -`snapshot -> press @e1 -> press @e2` can dispatch both presses if the first press takes no internal capture. -The second ref uses pre-action coordinates even if the first press navigated. -A false successful tap on a new screen is worse than an explicit stale-ref failure. +`snapshot -> press @e1 -> press @e2` can dispatch both presses when the first press performs no internal capture. +The second ref is resolved against pre-action coordinates, even if the first press navigated. +A false successful tap on the new screen is worse than a clear stale-ref failure. The lifetime contract must meet these requirements: @@ -70,39 +78,42 @@ The lifetime contract must meet these requirements: ## Decision -The terms below define the implemented model and appear in `CONTEXT.md`. -The coarse `snapshotRefsStale` marker is removed. -Read-only ref staleness derives from frame state. -A plain ref warns after frame expiry. -Read-only captures do not mark refs stale because they do not expire the frame. +The terms that this section introduces are now the implemented model. +The ref-frame vocabulary is promoted into `CONTEXT.md`. +The superseded coarse `snapshotRefsStale` marker is removed. +Read-only ref staleness now derives from the frame state: + +- A plain ref warns after the frame expires. +- A read-only capture no longer marks refs stale, because it does not expire the frame. ### Ref frames are separate from operational observations A session owns at most one **ref frame**. -This authorization namespace for mutation refs contains: +The frame is the authorization namespace for mutation refs. +It contains: -- A seeded monotonic **ref-frame epoch**, exposed as `refsGeneration` and accepted through `~s`. -- The immutable source tree required to resolve and guard issued refs. -- State `active` or `expired`. -- Issuance scope `all` or a bounded set of issued ref bodies. +- A seeded, monotonic **ref-frame epoch**. The existing `refsGeneration` field exposes the epoch. The existing `~s` input suffix accepts it. +- The immutable source tree that is necessary to resolve and guard the refs that the frame issued. +- A state of `active` or `expired`. +- An issuance scope of `all` or a bounded set of issued ref bodies. -Operational captures update the latest observation for selectors and diagnostics. +Operational captures update the latest observation that selector and diagnostic machinery use. They do not silently replace, reactivate, or reindex the authorized ref frame. -The implementation may share immutable tree objects between observation and frame. -It must not combine their lifetimes through mutable session-state aliases. +The implementation may share immutable tree objects between the observation and the frame. +But it must not make their lifetimes the same through aliases of mutable session state. Only one frame is retained. -A new active frame supersedes the previous frame and advances the epoch. -No historical generation-to-tree cache is introduced. -Complete and partial frames retain the immutable source tree. -Interaction guards and replay identity can depend on ancestors, siblings, overlays, and viewport context outside emitted nodes. -The issuance set bounds partial authority, not the evidence tree. -If frame and observation originate from the same capture, they share immutable capture data. +Activation of a new frame supersedes the previous frame and advances the epoch. +Historical generation-to-tree caches are not introduced. +Complete and partial frames retain the immutable source tree, because interaction guards and replay identity can depend on context outside the emitted subset. +This context includes ancestors, siblings, overlays, and the viewport. +The issuance set bounds partial authority; the evidence tree does not. +When the frame and the latest observation originate from the same capture, they share immutable capture data. Neither transition deep-copies the tree. The existing `snapshotGeneration` implementation evolves behind one ref-frame module. -Its former companion `snapshotRefsStale` is removed. -For wire compatibility, public `refsGeneration` and `@e12~s42` grammar remain unchanged. +The `snapshotRefsStale` marker that it originally paired with is now removed. +For wire compatibility, the public name `refsGeneration` and the `@e12~s42` grammar remain unchanged. ### Frame transitions @@ -116,27 +127,28 @@ For wire compatibility, public `refsGeneration` and `@e12~s42` grammar remain un | Sparse, failed, or unusable capture | None | Unchanged | Unchanged | | Session reopen | New random-seeded lifetime | Rejected until publication | Old lifetime rejected probabilistically as today | -Complete activation normally accompanies a result that exposes the stored frame's complete ref namespace. -This includes interactive or intentionally scoped snapshots. +A complete activation normally accompanies a command result that exposes the complete ref namespace for the stored frame. +This includes an interactive or intentionally scoped snapshot. An intentionally scoped snapshot replaces authorization with exactly its returned namespace. -It neither authorizes matching ref bodies outside the scope nor retains the old `all` scope. +It neither authorizes matching ref bodies outside that scope nor retains the old `all` scope. -Before a frame-activating capture uses an `@ref` scope, it must admit that ref against the active frame. +A frame-activating capture scoped by `@ref` must first admit that scope ref against the current active frame. Scoped output reindexes refs. -To preserve repeated scoped snapshots, the session retains one bounded **scope lineage** for consecutive read-only repetitions of that exact scope. -The lineage stores the admitted semantic scope as label/selector chain. -It stores neither the old positional tree nor mutation authority. -Repeated `snapshot -s @ref` may use lineage only if the immediately preceding frame publication was the same scoped capture. -These events clear lineage: +To preserve the existing contract for repeated scoped snapshots after this reindex, the session retains one bounded **scope lineage**. +The lineage is for consecutive read-only repetitions of that exact scope. +The lineage stores the admitted semantic scope (label/selector chain). +It does not store the old positional tree or mutation authority. +A repeated `snapshot -s @ref` may resolve through that lineage only when the immediately preceding frame publication was the same scoped capture. +Each of these events clears the lineage: - The first device-side effect. -- Unrelated frame publication. +- An unrelated frame publication. - Another scope. -Arbitrary stale or unissued scope refs fail closed. -They cannot use lineage or authorize another subtree. -Empty, sparse, failed, or unusable repetitions preserve the last useful frame and still-consecutive lineage. -Selector-scoped captures retain normal selector rules. +An arbitrary stale or unissued scope ref still fails closed. +It cannot borrow lineage or authorize a different subtree. +Empty, sparse, failed, or unusable repeated captures preserve the last useful frame and its still-consecutive lineage. +Selector-scoped captures follow their normal selector rules. These outputs are partial external publications: @@ -146,15 +158,17 @@ These outputs are partial external publications: - Settled diffs/tails. - Replay divergence screens. -They activate partial frames only if the caller or orchestrated workflow sees at least one ref and its epoch. -An empty partial result does not supersede useful authority. -Partial publication intentionally supersedes the prior frame. -This avoids unbounded generation-specific trees and makes authority explicit. -Only refs named in that response can authorize mutations, with the response-level epoch. +They activate a partial frame only when at least one ref and its epoch are visible to the caller or orchestrated workflow. +An empty partial result does not supersede useful existing authority. -An explicit non-diff complete snapshot leaf may activate a frame inside replay or batch before the outer orchestrator returns. -This remains true if the outer response later omits or digests the tree. -This execution-local exception preserves deterministic `snapshot -> @ref` scripts. +Publication of a partial frame intentionally supersedes the prior frame. +This avoids retention of an unbounded set of generation-specific trees. +It also makes the limited authority of the partial frame explicit. +Only refs named in that partial response can be used for mutation, and only with their response-level epoch. + +An explicit, non-diff complete snapshot leaf may activate a frame inside replay or batch before the outer orchestrator returns. +This is true even when the outer response level later elides or digests that tree. +This execution-local activation is a deliberate compatibility exception for deterministic `snapshot -> @ref` scripts. It authorizes only predeclared later bare-ref steps. It does not: @@ -163,39 +177,39 @@ It does not: - Add general result interpolation. Internally injected, cached, freshness, and recovery captures never receive this exception. -Nested non-empty partial publication may supersede scope. -However, batch and replay have no dynamic generation binding for a later step to consume it. +A nested non-empty partial publication may supersede scope. +But batch and replay have no dynamic generation binding with which a later step could consume it. ### Mutation admission and read behavior -A ref mutation is admitted only if all conditions hold: +A ref-targeting mutation is admitted only when all of these conditions hold: 1. The frame is active. -2. A pin names the current epoch, or a plain ref targets a complete `all` frame. -3. The ref body belongs to the issuance scope. +2. A pinned ref carries the current epoch, or a plain ref is used against a complete `all` frame. +3. The ref body belongs to the issuance scope of the frame. 4. Normal target guards and replay target-binding verification pass. -Resolution uses the authorized frame, never positional coincidence in a newer operational observation. +The target is resolved against the authorized frame, never against a newer operational observation by positional coincidence. Android freshness captures may verify current identity or provide comparison evidence. -They must not redefine the node authorized by an already-admitted `@eN`. +But they must not redefine which node an already-admitted `@eN` authorizes. -Read-only ref consumers remain fail-open with structured staleness warnings only if retained frame evidence can resolve the ref. +Read-only ref consumers remain fail-open with a structured staleness warning only when the retained frame contains the evidence necessary to resolve that ref. A stale read is observable and recoverable. -A stale mutation can affect the wrong element. -Missing evidence fails instead of using positional coincidence in a newer observation. -Reads never reactivate frames or give unissued refs mutation authority. +A stale mutation can act on the wrong element. +Missing frame evidence fails, instead of falling through to a newer operational observation by positional coincidence. +Reads never reactivate a frame or convert an unissued ref into mutation authority. Rejected mutations use the existing error system. -Admission evaluates typed reasons in this order: +Admission evaluates failures in this order, and each failure has a typed reason: 1. `ref_frame_expired`: a side effect invalidated the current frame. 2. `ref_generation_mismatch`: a pin names another epoch. 3. `plain_ref_requires_complete_frame`: a plain ref targets a partial frame. -4. `ref_not_issued`: a pin is outside a partial frame's issuance set. +4. `ref_not_issued`: a pinned ref is outside the issuance set of a partial frame. -These reasons distinguish a need for complete snapshot capture from a need for an emitted pinned ref. -Each failure includes `details.ref`, `details.reason`, `details.currentGeneration`, and frame scope. -Pinned inputs also include `details.mintedGeneration`. +With the typed reasons, callers can distinguish "capture a complete snapshot" from "use the emitted pinned ref". +Each failure includes `details.ref`, `details.reason`, `details.currentGeneration`, and the frame scope. +When the input is pinned, the failure also includes `details.mintedGeneration`. For example: ```text @@ -209,11 +223,11 @@ hint: Capture a fresh interactive snapshot or use a stable selector. ``` The daemon cannot truthfully report `mintedGeneration` for a plain ref. -If lifetime expiry caused failure, error messages must not claim that an existing ref was missing or lacked bounds. +When the actual failure is lifetime expiration, error messages must not claim that a found ref was missing or lacked bounds. ### Expiration occurs at the side-effect seam -Expiry occurs after all pre-action guards pass, including: +Expiration occurs after these and all other pre-action guards pass: - Syntax. - Capability. @@ -224,45 +238,47 @@ Expiry occurs after all pre-action guards pass, including: - Visibility. - Bounds. -It occurs immediately before the first operation that may change device-visible element identity. -Before awaiting that operation, persist the in-memory transition. +Expiration occurs immediately before the first operation that may change device-visible element identity. +Before you await that operation, persist the in-memory transition. -Pre-seam failure preserves the frame. -After the seam, success, timeout, cancellation, connection loss, and ambiguous platform errors all leave it expired. -The device may already have acted. -The seam also clears scoped-snapshot lineage. -No success-only rollback occurs. +Failure before this seam preserves the frame. +After the seam is crossed, success, timeout, cancellation, connection loss, and ambiguous platform errors all leave the frame expired, because the device may already have acted. +A crossing of the seam also clears scoped-snapshot lineage. +There is no success-only rollback. -This rule covers every execution shape: +This rule applies to every execution shape: - Runtime ref/selector/coordinate paths expire before `tap`, `fill`, `longPress`, text input, or gesture execution. -- Native web ref paths perform ref preflight, then expire before `clickRef` or `fillRef`. -- Direct iOS selector paths consume no ref but expire existing frames before the fused runner resolve-and-act request. -- Platform-generic, app lifecycle, keyboard, alert, settings, and provider actions expire at their actual leaf side-effect seam. - They do not expire at generic admission. -- Existing-session `open`/relaunch expires that session's frame before first close/launch dispatch. - Post-dispatch failure or timeout leaves it expired. - Successful `close` deletes the session, with no frame to restore. -- After an error, fallback or retry may proceed only if the error proves no prior side effect. +- Native web ref paths run their existing ref preflight first, and then expire before `clickRef` or `fillRef`. +- Direct iOS selector paths consume no ref, but they expire any existing frame before the fused runner resolve-and-act request. +- Platform-generic, app lifecycle, keyboard, alert, settings, and provider actions expire at their actual leaf side-effect seam, not at generic request admission. +- `open`/relaunch against an existing session expires the frame of that session before the first close/launch dispatch. + A failure or timeout after dispatch leaves the frame expired. + A successful `close` deletes the session, and thus retains no frame to restore. +- After an error, a fallback or retry may proceed only when the error proves no prior side effect. Ambiguous post-dispatch errors neither restore the frame nor authorize an automatic second action. -- Android blocking-dialog recovery mutates the device. - Before recovery tap/relaunch, it must cross the same idempotent seam, even during apparent readiness work. - If recovery mutates before a requested ref action dispatches, that action aborts with `ref_frame_expired`. - It cannot continue against the recovered UI. - Selector and coordinate actions may re-resolve and continue under existing policies. +- Android blocking-dialog recovery is itself device-mutating. + Its recovery tap/relaunch must cross the same idempotent seam before it acts, even when apparent readiness work invokes the recovery. + If recovery mutates before a requested ref action dispatches, that ref action aborts with `ref_frame_expired`. + The ref action cannot continue against the recovered UI. + Selector and coordinate actions may re-resolve and continue under their existing policies. + +The seam is deliberately inside leaf execution. +Expiration at router entry would cause these problems: -The seam intentionally belongs inside leaf execution. -Router-entry expiry would reject the current command's ref, invalidate ordinary validation failures, and mishandle multiplexed or orchestrating commands. +- It would reject the ref that the current command needs. +- It would invalidate on ordinary validation failures. +- It would mishandle multiplexed and orchestrating commands. -Some operations combine final status/target checks and mutation in one request. -Examples include direct iOS selectors and current native alert or keyboard actions. -Until a lower-level “will act” callback exists, fused-request dispatch is the conservative seam. -Later `not found`, `already hidden`, timeout, or no-alert results do not restore the frame. -A separate definitive read-only status request preserves the frame if it proves no mutation request was dispatched. +Some platform operations fuse their final status/target check and their mutation in one request. +Examples are direct iOS selectors and current native alert or keyboard actions. +Until a lower-level "will act" callback exists, the dispatch of that fused request is the conservative seam. +A later `not found`, `already hidden`, timeout, or no-alert result is post-seam and does not restore the frame. +A command with a separate, definitive read-only status request preserves the frame when that status proves that no mutation request was dispatched. ### Command effects are daemon-owned and completely classified -The command descriptor's daemon facet declares request policy: +The daemon facet of the command descriptor declares a request policy: ```ts type RefFrameEffect = 'preserve' | 'may-invalidate' | 'delegated'; @@ -272,77 +288,84 @@ type DaemonRefFrameEffect = RefFrameEffect | ((request: DaemonRequest) => RefFra The daemon registry exposes a named resolver. Every daemon command is classified. -Classification checks honesty/completeness, not transitions. -`may-invalidate` means some successful path can cross a device side effect. -Only after that path is selected does the leaf call the ref-frame module. -Request-sensitive commands use a focused resolver instead of one classification for all subcommands. +But the classification is an honesty/completeness guard, not the transition site. +`may-invalidate` means that some successful path can cross a device side effect. +The leaf implementation still calls the ref-frame module only when that path is selected. +Request-sensitive commands use a focused resolver, instead of a pretense that all subcommands behave alike. The registry is the exhaustive source of truth. -This non-exhaustive table guides representative classifications. -It must not become a second prose registry: +The table below is non-exhaustive guidance for the classification of representative actions. +The table must not become a second prose registry: | Effect | Commands/actions | | --- | --- | | `may-invalidate` | press, click, fill, longpress, type, focus, scroll, swipe, gesture, back, home, `tv-remote`, orientation, open/relaunch, trigger/push delivery, settings changes, install/reinstall, React Native overlay dismissal, and lifecycle operations that can replace the visible surface | -| Conditional resolver | keyboard status preserves; dismiss/return/input invalidate. Alert get/wait preserve; accept/dismiss invalidate. Find reads preserve; click/fill/focus/type delegate to leaf mutation. | +| Conditional resolver | keyboard status preserves while dismiss/return/input invalidate; alert get/wait preserve while accept/dismiss invalidate; find reads preserve while click/fill/focus/type delegate to their leaf mutation | | `delegated` | batch, replay, and test/suite orchestrators; each nested leaf owns its transition | -| `preserve` | snapshots and other observation, assertion, screenshot, recording, trace, logs, events, network inspection, performance, inventory, capability, lease, and transport-management operations, unless a selected subaction directly manipulates the visible surface | +| `preserve` | snapshots and other observation, assertion, screenshot, recording, trace, logs, events, network inspection, performance, inventory, capability, lease, and transport-management operations unless a selected subaction directly manipulates the visible surface | -Clipboard reads and writes preserve the frame because pasteboard state alone does not change element identity. -A later paste/type action invalidates independently. +Clipboard reads and writes preserve the frame, because pasteboard state alone does not change element identity. +A later paste/type action is independently invalidating. -Generic routes have no exception. -`back`, `home`, `orientation`, `scroll`, `tv-remote`, and `app-switcher` must cross the transition in the generic daemon leaf when they act. -The completeness gate covers every daemon-projected command, including generic fallbacks. +Generic routing is not an exception to the policy. +`back`, `home`, `orientation`, `scroll`, `tv-remote`, and `app-switcher` all reach the generic daemon leaf. +When they act, they must cross the same transition there. +The completeness gate covers every command projected to the daemon, including generic fallbacks. Thus, a missing facet cannot hide an unclassified mutation. -Mutations from unrelated external tools remain outside this session guarantee. +Mutations that unrelated external tools perform remain outside this session guarantee. -Runner-side classification does not define this policy. -Apple runner traits govern retry eligibility, launch policy, and recorded-failure conversion at a lower wire-command boundary. -The TypeScript `readOnly` trait controls: +This policy is not derived from the runner-side classification. +Apple runner command traits govern retry eligibility, launch policy, and the recorded-failure conversion at a lower wire-command seam. +The TypeScript `readOnly` trait is itself an input to several daemon decisions: - Read-only resend. - Session-invalidation skip. - Transport error classification. - Readiness preflight. -`refFrameEffect` governs daemon session authorization, including commands outside the Apple runner. +`refFrameEffect` governs daemon session authorization, and it includes commands that never reach the Apple runner. Narrow consistency tests may cover direct mappings. -Blanket parity would combine distinct concepts. +But blanket parity would couple different concepts. The existing per-session request lock serializes frame admission and transitions. The frame is shared session state, not per-client or per-lease history. -Generation pins identify rejected epochs and reasons precisely. -They neither reserve frames nor prevent another caller's mutation from expiring them. +Generation pins make the rejected epoch and reason precise. +They do not reserve a frame, and they do not prevent the mutation of another caller from expiring it. No second ref-specific mutex is introduced. ### Batch, replay, recording, and repeated actions -Batch and replay invoke sequential steps through ordinary daemon dispatch and declare `delegated`. -Outer command start never expires a frame. -Frame state passes naturally between leaves: +Batch and replay invoke their steps sequentially through ordinary daemon dispatch, and they are `delegated`. +They never expire a frame merely because the outer command began. +Frame state naturally carries between leaf steps: - `snapshot -> press @e1 -> press @e2` executes the first press and rejects the second. -- `snapshot -> press @e1 -> press @e1 -> press @e2` stops on the second press, before `@e2`. +- `snapshot -> press @e1 -> press @e1 -> press @e2` also stops on the second press, before `@e2`. - `snapshot -> press @e1 -> snapshot -> press @e2` is admitted. - Selector-only and coordinate-only batches retain their request-round-trip advantage. -- Stale-ref failure stops batch under existing policy. - It preserves `step`, `command`, `executed`, `total`, `partialResults`, diagnostics, and `details.reason`. +- A stale-ref failure stops batch through its existing policy. + The failure preserves `step`, `command`, `executed`, `total`, `partialResults`, diagnostic fields, and `details.reason`. -Batch gains no nested MCP pin traversal, intermediate-result interpolation, unsafe ref-stability override, or second static lifecycle interpreter. -Normal leaf dispatch is the sole enforcement path. +Batch does not gain any of these: + +- Nested MCP pin traversal. +- Intermediate-result interpolation. +- An unsafe "assume stable refs" override. +- A second static lifecycle interpreter. + +Normal leaf dispatch is the single enforcement path. Recorded `.ad` scripts remain selector-first. -The recorder converts successfully resolved refs to selector chains. -It refuses to persist unresolved session-bound bare refs. -Generation suffixes remain runtime evidence and are removed from durable recordings because seeded epochs are not portable across sessions. -Legacy handwritten `.ad` scripts that mutate several bare refs from one snapshot become intentionally incompatible. -They must capture between mutations or use selectors. -Replay target-binding verification finishes before the seam. -Identity divergence preserves the frame. -Post-dispatch action failure leaves it expired. +The recorder continues to convert successfully resolved refs to selector chains. +It refuses to persist an unresolved session-bound bare ref. +Generation suffixes remain runtime evidence. +They are stripped from durable recordings, because seeded epochs are not portable across sessions. +Legacy hand-written `.ad` scripts that mutate through several bare refs from one snapshot become intentionally incompatible. +These scripts must capture between mutations or use selectors. +Replay target-binding verification completes before the side-effect seam. +Identity divergence preserves the frame, while an action failure after dispatch leaves it expired. -These repetitions form one logical command: +Each of these is one logical command: - `press --count`. - Double-tap. @@ -351,174 +374,176 @@ These repetitions form one logical command: - `swipe --count`. - Platform sequence chunks. -They resolve or plan once and expire once before the first repetition. -Every repetition executes the original physical point/plan. -They neither re-resolve the target nor reactivate the frame between repetitions. -Later repetition or chunk failure leaves the frame expired. -This preserves explicit repeated physical actions while preventing frame reuse by later commands. +Each one resolves or plans once, expires once before the first repetition, and executes the original physical point/plan for every repetition. +It does not re-resolve the target or reactivate the frame between repetitions. +Failure in any later repetition or chunk leaves the frame expired. +This preserves the explicit repeated-physical-interaction contract, and it prevents later commands from reuse of the frame. ### Partial issuance and MCP -`find`, settled diffs/tails, and replay divergence screens publish only returned refs. -They must not mark the whole frame as issued. +`find`, settled diffs/tails, and replay divergence screens publish only the refs that they actually return. +They must not call a whole-frame "refs issued" marker. Read-only `find` may partially publish its returned ref. -Mutating `find` returns its acted ref only as diagnostic pre-action identity. -It must omit `refsGeneration`. -It must not publish or activate a frame. -It cannot validate that ref after the nested action. - -The MCP executor retains per-ref pins by state-directory/session scope. -Issuance must distinguish the selected `find` action instead of command name alone. -Read-only `find` with `refsGeneration` merges its partial pin. -Mutating `find` without `refsGeneration` is explicitly non-issuing and leaves remembered pins unchanged. -It must not enter a generic missing-generation branch that clears the session's pin scope. -The executor must carry sufficient normalized command/action context for this distinction. -Mutation does not clear remembered pins. -The old epoch lets the daemon return a precise stale rejection. -MCP neither traverses nested batch input refs nor learns refs from nested batch results. -Both remain explicit non-goals. - -Response-level `refsGeneration` retains authority without repeated per-node bytes. -Snapshot nodes never gain generation bytes. -Every reusable partial CLI text ref must render as ready-to-copy `@eN~s`. -JSON and Node.js retain plain ref bodies with one response-level generation. -Before mutation, callers must pair them. -MCP may render plain refs because it stores and forwards pins internally. -Complete snapshot trees remain plain to preserve the token budget. -A mutating `find` diagnostic ref never renders or stores as pinned because it has no `refsGeneration`. +A mutating `find` returns the acted ref only as diagnostic pre-action identity: + +- It must omit `refsGeneration`. +- It must not publish or activate a frame. +- It cannot make that ref valid after the nested action. + +The MCP executor continues to store per-ref pins by state-directory/session scope. +Issuance handling must distinguish the selected `find` action. +It must not infer issuance from the command name alone. +A read-only `find` with `refsGeneration` merges its partial pin. +A mutating `find` without `refsGeneration` is explicitly non-issuing, and it leaves every remembered pin unchanged. +It must not enter a generic missing-generation branch that clears the pin scope of the session. +The executor must carry sufficient normalized command/action context to make that distinction. +Remembered pins are not cleared on mutation. +The daemon produces a precise stale rejection from the remembered old epoch that the executor sends. +MCP currently does not traverse refs nested inside batch inputs, and it does not learn refs from nested batch results. +This remains an explicit non-goal. + +The response-level `refsGeneration` field remains the token-economical authority. +Snapshot nodes never gain per-node generation bytes. +Every reusable ref in a partial CLI text result must render in the ready-to-copy `@eN~s` form. +JSON and Node.js responses retain plain ref bodies plus one response-level generation. +Before a mutation, callers must pair them. +MCP may render a plain ref, because it stores and forwards the pin internally. +Complete snapshot trees remain plain to protect the token budget. +A mutating `find` diagnostic ref is never rendered or stored as pinned, because it carries no `refsGeneration`. ## Performance and compatibility -Expiry is O(1): one monotonic epoch transition and bounded in-memory state. +Expiration is O(1): one monotonic epoch transition and bounded in-memory state. It adds no capture, platform request, daemon round trip, or per-node output. -At most one authorized frame and one latest operational observation remain retained. -They share immutable capture objects without deep copies. -The model never caches unbounded full-tree history. +The model retains at most one authorized frame and one latest operational observation. +These share immutable capture objects, not deep copies. +The model never caches an unbounded history of full trees. Existing capture size limits remain the bound. -Memory pressure must not silently expire or replace frames. -Otherwise, deterministic admission would depend on the environment. +Memory pressure must not silently expire or swap a frame, because that would make deterministic admission depend on the environment. -The intentional cost affects workflows. -Several ref mutations may require explicit observations between them. -Callers can avoid unnecessary captures through: +The real cost is at the workflow level, and it is intentional. +A ref-oriented sequence that performs several mutations may need an explicit observation between them. +Callers can avoid unnecessary captures with: - Selectors. - Direct selector fast paths. -- A pinned emitted ref from settled partial output. -- `--count` for repeated physical actions at one target. +- A settled partial result whose emitted ref is pinned. +- `--count` for a repeated physical action at one resolved target. + +The daemon never captures automatically for the sole purpose of a ref renewal. + +Older clients remain wire-parse-compatible: -The daemon never auto-captures only to renew refs. +- Command grammar, `refsGeneration`, and `~s` remain unchanged. +- Structured error details are additive. -Older clients remain wire-parse-compatible. -Command grammar, `refsGeneration`, and `~s` remain unchanged. -Structured error details are additive, so ADR 0006 requires no protocol increment. -Behavioral compatibility is incomplete: +Thus, ADR 0006 requires no protocol bump. +But older clients are not fully compatible in behavior: -- Unsafe plain-ref mutation chains now fail. -- Fused commands can expire frames despite eventual no-op results. - Examples include already-hidden `keyboard dismiss` and alert actions with no alert. -- Read-only refs without retained evidence fail instead of warning and resolving by positional coincidence in newer observations. +- Unsafe plain-ref mutation chains newly fail. +- A fused command can also expire the frame when its eventual result is a no-op. + Examples are `keyboard dismiss` that reports an already hidden keyboard, and an alert action that reports no alert. +- Read-only refs that lack retained frame evidence newly fail, instead of a warning and a resolution by positional coincidence against a newer observation. These changes require CLI/help, changelog, `.ad` migration, and batch guidance. -This contract composes with existing ADRs: +This decision composes with existing ADRs as follows: -- ADR 0003/0008: daemon-owned effect policy composes in the descriptor daemon facet and projects only through the daemon registry. -- ADR 0004: capture plans and sparse-quality rules remain unchanged, with no added capture. -- ADR 0005: runner liveness/preflight remains independent of ref authorization. -- ADR 0006/0010: no RPC increment; typed `details.reason` and actionable hints follow the error contract. -- ADR 0007: frames use the existing session/request lock, outside client/lease-local state. -- ADR 0009: Apple, Android, Linux, and web share one target model with projected public platform output. -- ADR 0011: lifetime spans session sequences instead of another element-guarantee cell. +- ADR 0003/0008: daemon-owned effect policy is composed in the descriptor daemon facet and projected only through the daemon registry. +- ADR 0004: capture plans and sparse-quality rules are unchanged, and no capture is added. +- ADR 0005: runner liveness/preflight state remains independent from session ref authorization. +- ADR 0006/0010: there is no RPC bump; typed `details.reason` and actionable hints follow the error contract. +- ADR 0007: the frame is session-scoped under the existing session/request lock, not client-local or lease-local. +- ADR 0009: one target model applies to Apple, Android, Linux, and web, while public platform output remains projected. +- ADR 0011: lifetime is a sibling session-sequence invariant, not another path-by-element-guarantee cell. Every mutating path still crosses the shared seam. -- ADR 0012: durable target identity retains selectors/evidence; divergence refs issue bounded partial authority. +- ADR 0012: durable target identity remains selector/evidence based, and divergence refs become honest partial issuance. - ADR 0013: coordinate gestures expire frames but remain outside the element guarantee matrix. ## Consequences -- Stale refs cannot silently mutate a different element after navigation. - Typed failure requires a fresh observation or selector. -- Multiple logical ref mutations require captures between them, issued settled refs, selectors, or one repeated-action command such as `--count`. - Legacy handwritten `.ad` scripts that reuse bare refs from one snapshot must change. -- Conservative fused requests may consume authority even if the platform reports no required action. - This prioritizes prevention of wrong-screen actions over ref preservation after ambiguous dispatch. -- Reads without retained evidence fail instead of using positional coincidence in newer trees. - Reads with retained evidence preserve structured warnings. -- Consecutive scoped snapshots retain admitted semantic scope. - Mutation, unrelated publication, or another scope clears lineage. -- Bounded session-memory bookkeeping adds no automatic capture, platform call, round trip, or per-node response bytes. +- A stale ref cannot silently mutate a different element after navigation. + The caller receives a typed failure and must choose a fresh observation or a selector. +- Ref-oriented workflows that perform multiple logical mutations must capture again between the mutations, consume an honestly issued settled ref, use selectors, or use one repeated-action command such as `--count`. + Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must change. +- Conservative fused requests may consume ref authority even when the platform later reports that no action was necessary. + This favors the avoidance of a wrong-screen action over the preservation of refs after an ambiguous dispatch. +- A read without retained evidence fails, instead of resolving the same positional ref body in a newer tree. + Reads with retained evidence keep the structured warning behavior. +- Consecutive repeated scoped snapshots retain their admitted semantic scope. + Any mutation, unrelated publication, or different scope breaks that lineage. +- The implementation pays for bounded bookkeeping in session memory. + But it adds no automatic capture, platform call, round trip, or per-node response bytes. ## Required evidence -Completion requires contract and provider tests that prove: +The implementation is not complete until contract and provider tests prove all of these: -1. After an unobserved first mutation, bare and pinned second mutations reject. +1. After an unobserved first mutation, a bare and a pinned second mutation reject. 2. Definitive pre-seam validation, target, and replay-identity failures preserve the frame. -3. Timeout, cancellation, connection loss, and other post-seam failures leave it expired. -4. Every listed path crosses the same seam: runtime ref/selector/coordinate, native ref, direct selector, provider-backed interaction, and provider-backed lifecycle/generic paths. -5. Android freshness cannot retarget admitted refs by positional coincidence. -6. Read-only operational captures do not replace authorized frames. -7. Partial publication admits only emitted pinned refs and reports each typed failure. - It cannot authorize unrelated plain refs. - CLI text pins every reusable partial ref. - JSON and Node.js retain response-level representation. -8. Complete publication admits its plain namespace. - Sparse/unusable captures do not. -9. Mutating `find` omits `refsGeneration` and never pins or activates its pre-action ref as post-action state. - After MCP remembers a snapshot pin and performs mutating `find`, later ref input still forwards the old pin. - The daemon rejects it as stale. -10. Batch and replay delegate per step and retain failure/partial-result contracts. - They permit `snapshot -> @ref` with an intermediate digest without exposed pins or dynamic bindings. -11. Recorded scripts contain selectors instead of portable generation claims. +3. Timeout, cancellation, connection loss, and other post-seam failures leave the frame expired. +4. These paths cross the same seam: runtime ref/selector/coordinate, native ref, direct selector, provider-backed interaction, and provider-backed lifecycle/generic paths. +5. Android freshness cannot retarget an admitted ref through positional coincidence. +6. Read-only operational captures do not replace the authorized frame. +7. Partial publication admits only emitted pinned refs, reports each typed admission failure, and cannot bless unrelated plain refs. + CLI text renders every reusable partial ref with its pin, while JSON and Node.js retain the response-level representation. +8. Complete publication admits its plain namespace, while sparse/unusable captures do not. +9. Mutating `find` omits `refsGeneration` and never pins or activates its pre-action ref as current post-action state. + An MCP sequence that first remembers a snapshot pin and then performs mutating `find` still forwards the remembered old pin on a later ref input. + Thus, the daemon rejects that pin as stale. +10. Batch and replay delegate per step and retain their existing failure/partial-result contracts. + They allow `snapshot -> @ref` with an intermediate digest, without exposed pins or dynamic bindings. +11. Recorded scripts contain selectors, not portable generation claims. 12. Press/swipe series and Apple chunks resolve and expire exactly once. -13. Android dialog recovery expires before its first side effect. - After recovery mutates, it aborts an outstanding ref action. -14. Consecutive `snapshot -s @ref` calls resolve the same admitted semantic scope after ref reindex. - Mutation, unrelated publication, or another scope clears lineage. - Arbitrary stale refs cannot use it. - Empty/unusable repetitions preserve the last useful frame and still-consecutive lineage. -15. Fused keyboard/alert and split status/action requests follow their conservative seams, including already-hidden and no-alert outcomes. -16. Failed or timed-out existing-session open/relaunch leaves the old frame expired after dispatch. -17. Read-only refs with retained stale evidence return structured warnings. - Missing evidence fails without positional fallback to newer observations. -18. A large-tree fixture proves that frame/observation separation adds no deep copy. - It records bounded peak retained nodes/bytes. +13. Android dialog recovery expires before its first recovery side effect. + After recovery mutates, the recovery aborts an outstanding ref action. +14. Consecutive repeated `snapshot -s @ref` calls resolve the same admitted semantic scope after scoped output reindexes refs. + A mutation, an unrelated publication, or a different scope breaks the lineage. + An arbitrary stale ref cannot use the lineage. + Empty/unusable repetition preserves the last useful frame and the still-consecutive lineage. +15. Fused keyboard/alert requests and split status/action paths follow their documented conservative seams, including already-hidden and no-alert outcomes. +16. A failed or timed-out existing-session open/relaunch leaves the old frame expired after dispatch. +17. A read-only ref with retained stale evidence returns the structured warning. + Missing frame evidence fails, without a fall-through to a newer observation by positional coincidence. +18. A large-tree fixture proves that frame/observation separation introduces no deep copy. + The fixture records the bounded peak of retained nodes/bytes. 19. Enforcement and error shapes are consistent across supported platforms. -Performance regression tests prove that admission, expiry, and frame/observation separation add no implicit capture or provider/platform call. -Device benchmarks are required only if tests or live evidence reveal regression. -No path may claim success until a test proves execution of the relevant direct/native/runtime/provider branch. +Performance regression tests prove that admission, expiration, and frame/observation separation add no implicit capture or provider/platform call. +Device benchmarks are required only if those tests or live evidence reveal a regression. +No path may claim success until a test proves that the relevant direct/native/runtime/provider branch actually executed. -Before platform enforcement, fresh-build live evidence must exercise every affected supported production seam: +Before enforcement is enabled for a platform, fresh-build live evidence must exercise every supported production seam affected on that platform: -- Apple runtime-ref and direct/native paths where available. +- Apple runtime-ref and direct/native paths, where available. - Android helper freshness and recovery. - Generic/lifecycle mutation. -- At least one real provider-backed interaction and lifecycle operation. +- At least one real provider-backed interaction plus lifecycle operation. -Every run must prove: +Each run must prove all of these: 1. A fresh ref succeeds. -2. A second mutation with the stale ref rejects before dispatch. +2. A second mutation with the stale ref is rejected before dispatch. 3. A fresh observation restores usability. -Unexercised seams remain disabled or explicit release blockers. -Fixture tests and registry claims are necessary but do not replace live evidence. +A seam that is not exercised remains disabled or is recorded as an explicit release blocker. +Fixture-backed tests and registry claims are necessary, but they do not substitute for this live evidence. ## Alternatives considered -- **Stable refs across trees.** Rejected because positional snapshots have no durable cross-tree identity. +- **Stable refs across trees.** Rejected because positional accessibility snapshots have no durable cross-tree identity. Semantic IDs are neither universal nor unique. -- **Expire only after successful mutation.** Rejected because the device may act before timeout or transport failure. -- **Use one snapshot for observation and authorization.** Rejected because internal read-only captures can reindex or invalidate caller refs. -- **Automatically capture after every mutation.** Rejected because it adds the dominant platform cost and can fail or return sparse output. - It hides lifetime instead of defining it. -- **Retain every generation's full tree.** Rejected as unbounded memory unnecessary for one current frame. -- **Immediately add per-ref history.** Rejected until evidence requires concurrent generations. - One bounded current frame and issuance scope suffice. -- **Derive policy from runner traits.** Rejected because runner traits classify wire commands for remaining app work. - Daemon ref authorization classifies their effects on the frame. -- **Add batch interpolation or unsafe ref-stability overrides.** Rejected as new orchestration interfaces that bypass the safety rule. -- **Put lifetime in ADR 0011's element path matrix.** Rejected because lifetime spans commands, time, non-element invalidators, and orchestrators. - The matrix retains per-path element guarantees. +- **Expire only after successful mutation.** Rejected because a timeout or transport failure may occur after the device acted. +- **Keep one snapshot for observation and authorization.** Rejected because read-only internal captures can reindex or invalidate caller refs. +- **Automatically capture after every mutation.** Rejected because it adds the dominant platform cost, and it can itself fail or be sparse. + It also hides lifetime, instead of defining it. +- **Retain every generation's full tree.** Rejected because it uses unbounded session memory. + It is also unnecessary for a single-current-frame contract. +- **Add a per-ref historical ledger immediately.** Rejected until evidence requires concurrent generation support. + One bounded current frame plus issuance scope is sufficient. +- **Derive the policy from the runner's command traits.** Rejected because those traits classify a wire command for what the runner may still do about its app. + Daemon ref authorization classifies a command for what the command did to the frame. +- **Add batch interpolation or an unsafe ref-stability override.** Rejected as a new orchestration interface that bypasses the same safety rule. +- **Force lifetime into ADR 0011's element path matrix.** Rejected because ref lifetime spans commands, time, non-element invalidators, and orchestrators. + The matrix remains the enforcement record for per-path element guarantees. From e18eb3218435b5f9488cf46e30b57b3ec4a112e9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 44/55] docs(adr): restore the frozen meaning in ADR 0015 Restore technical terms that code uses: bug class, bug-class flows, settle ordering, settle-loop ordering. Restore "never authoritative" for observation frames, and the "as long as" condition on ADR 0013 internal rewrites. Restore dropped facts: the oracle replaced the hand-typed fixture, Layer 3 is a small set, documented deviations are not silent mismatches, a unit test rather than a device scenario pins percent truncation. Make the glance rules match the body: inline-settle exception, ambiguity rule under `optional`, failed-dispatch invalidation. --- docs/adr/0015-direct-maestro-engine.md | 348 +++++++++++++------------ 1 file changed, 178 insertions(+), 170 deletions(-) diff --git a/docs/adr/0015-direct-maestro-engine.md b/docs/adr/0015-direct-maestro-engine.md index bcbcebb2f6..68a142c2b5 100644 --- a/docs/adr/0015-direct-maestro-engine.md +++ b/docs/adr/0015-direct-maestro-engine.md @@ -6,248 +6,256 @@ Accepted ## Rules at a glance -- Parse supported YAML into a source-preserving typed Maestro program. Execute it through a narrow compatibility runtime port. -- Do not route Maestro programs through `SessionAction`, private command names, positional JSON, or recursive replay dispatch. -- Keep variables and observation generation in one execution context. Every mutation attempt invalidates retained evidence before dispatch. -- Keep geometry action-local. Do not reuse it across command boundaries. -- Preserve upstream first-match/index semantics and the stated public-command ambiguity exception. -- Defer pending stabilization to the next mutation. Authored observations poll their own conditions immediately. -- Preserve the performance contract and declare conformance divergences. -- Keep `${...}` interpolation lookup-only. Gate `evalScript` through `MaestroEngineOptions.trustedScripts`. +- Parse supported Maestro YAML into a source-preserving typed program. Execute it directly through a narrow compatibility runtime port. +- Do not lower the program through `SessionAction`, private command names, positional JSON, or a recursive replay dispatcher. +- One explicit execution context owns variables and the current observation generation. +- Every mutating attempt invalidates retained evidence before dispatch, including an attempt whose dispatch reports failure. +- Keep geometry action-local. Never carry it across command boundaries. +- Preserve upstream first-match and explicit-index semantics. Do not adopt the unique-match requirement of the public command surface. +- Maestro `optional` does not suppress an `AMBIGUOUS_MATCH` from a nested public command. +- Before another mutating command, settle the pending stabilization boundary. Commands with specialized completion semantics settle inline instead. +- An authored observation after a mutation skips the pending barrier and polls its own condition immediately. +- Preserve the performance contract. Encode documented intentional deviations as declared expected-divergence entries. +- Keep `${...}` interpolation variable-lookup-only. Gate `evalScript` with `MaestroEngineOptions.trustedScripts`. - Do not ship two production engines or a runtime fallback between them. ## Context -At decision time, Maestro YAML compiled to generic `SessionAction` strings. -Commands incompatible with native replay became private `__maestro*` names with positional payloads. -Replay dispatch routed those strings through a second Maestro switch, then recursively invoked ordinary daemon commands. -Replay variable scope and three `WeakMap` caches divided compatibility state. +At decision time, Maestro YAML compiled into generic `SessionAction` strings. +Commands whose semantics did not match native replay became private `__maestro*` names with positional payloads. +Replay dispatch then routed those strings through a second Maestro switch before it recursively invoked ordinary daemon commands. +Compatibility state was split between the replay variable scope and three `WeakMap` caches. -This path acquired a second selector resolver, several polling loops, action-after-assertion recovery, and gesture-coordinate adapters. +That path grew a second selector resolver, several polling loops, action-after-assertion recovery, and gesture-coordinate adapters. The indirection makes successful responses hard to prove. -For example, a fast native wait can prove text existence while compatibility requires a visible Maestro target. -Positional decode also hides authored coordinate space until runtime. +For example, a fast native wait may establish text existence while the compatibility assertion requires a visible Maestro target. +The indirection also hides authored coordinate space until runtime positional decoding. -The incoming implementation had approximately 5,000 production lines and 5,000 focused test lines. -Android compatibility is materially faster than native Maestro in pager and navigation suites. -This advantage is a product constraint, not spare migration capacity. +The inbound implementation was approximately 5,000 production lines and 5,000 focused test lines. +Android compatibility is materially faster than native Maestro in the pager and navigation suites. +That advantage is a product constraint, not expendable migration headroom. -ADR 0013 owns public gesture normalization, contact topology, trajectory plans, and native injection. -Maestro supports only single-pointer swipe gestures. -These normalize to canonical `pan` with origin, delta, and durationMs. -Daemon-internal `internal.gestureExecutionProfile` carries `endpoint-hold`. -This preserves iOS fast-swipe-then-hold behavior without a public execution-profile option. +ADR 0013 separately owns public gesture normalization, contact topology, trajectory planning, and native injection. +Maestro's supported gesture surface is single-pointer swipe only. +Maestro swipes normalize to ADR 0013's canonical `pan` input (origin, delta, durationMs). +They carry the `endpoint-hold` execution profile through a daemon-internal compatibility seam (`internal.gestureExecutionProfile`). +This preserves iOS fast-swipe-then-hold behavior, and the public command surface does not expose the profile. ## Decision Parse supported Maestro YAML into a source-preserving typed program. -Execute it directly through a narrow compatibility runtime port. -Do not lower it through `SessionAction`, private command names, positional JSON, or recursive replay dispatch. +Execute the program directly through a narrow compatibility runtime port. +Do not lower the program through `SessionAction`, private command names, positional JSON, or a recursive replay dispatcher. The daemon adapter may invoke ordinary public commands as typed operations. -Those calls reuse shared command semantics without re-entry into Maestro parsing or compatibility dispatch. +Those calls reuse the shared command semantics and never re-enter Maestro parsing or compatibility dispatch. The engine has five responsibilities: -1. The parser validates supported grammar and preserves source path and line on every command. -2. The interpreter owns hooks, includes, environment scopes, conditions, repeat/retry, and ordered execution. - Runtime repeat/retry blocks remain compact plan nodes without expansion of authored iteration counts. +1. The parser validates the supported grammar and preserves source path and line on every command. +2. The interpreter owns hooks, includes, environment scopes, conditions, repeat/retry, and ordered command execution. + Runtime repeat/retry blocks remain compact plan nodes. + The plan does not expand authored iteration counts. `repeat.times` preserves the authored count. - Retry clamps `maxRetries` to upstream's three-retry preset, for four total attempts. + Retry clamps `maxRetries` to upstream's three-retry preset (four total attempts). 3. The runtime port exposes typed app, input, observation, target, and single-pointer gesture operations. - Existing agent-device runtimes and platform adapters execute them. -4. One explicit execution context owns variables and current observation generation. - Mutation invalidates that generation. + The existing agent-device runtime and platform adapters back these operations. +4. One explicit execution context owns variables and the current observation generation. + A mutation invalidates that generation. Reads may reuse semantic evidence only within the same generation. - Geometry stays action-local and never crosses command boundaries. -5. An observer adapts source-aware progress, traces, artifacts, and failures to existing replay/test results. + Geometry is action-local and is never carried across command boundaries. +5. An observer adapts source-aware progress, traces, artifacts, and failures to the existing replay and test result contracts. Observer telemetry is redacted and best-effort. Trace persistence cannot change command success or failure. -Maestro `stopApp` uses a daemon-internal app-only close and preserves the owning replay session. -Public `close` remains the session finalizer for recordings, runner state, leases, and stored runtime hints. +Maestro `stopApp` terminates its target through a daemon-internal app-only close and preserves the owning replay session. +Public `close` remains the session finalizer that releases recordings, runner state, leases, and stored runtime hints. The engine does not implement platform input. -Absolute and percentage swipes preserve authored endpoints without hierarchy capture. +Absolute and percentage swipes preserve authored endpoints without a hierarchy capture. Directional horizontal swipes reuse ADR 0013's shared in-page preset geometry. Thus, an iOS right swipe does not become an interactive-back gesture. Vertical presets retain Maestro's platform geometry. -Every viewport-relative swipe resolves the least expensive fresh interaction viewport available. -ADR 0013 then validates every planned sample. -The nested public `gesture` request pairs that viewport with `internal.gestureExecutionProfile: 'endpoint-hold'`. -This produces iOS fast-swipe-then-hold behavior without public `executionProfile`. -If normalization already resolved a viewport, the adapter supplies that exact frame as daemon-internal metadata. -ADR 0013 consumes it without another platform probe. +All viewport-relative swipes resolve the least expensive fresh interaction viewport available, so that ADR 0013 can validate every planned sample. +Maestro swipes pair that resolved viewport with `internal.gestureExecutionProfile: 'endpoint-hold'` on the nested public `gesture` request. +Thus, ADR 0013 produces the iOS fast-swipe-then-hold profile without a public `executionProfile` field. +If normalization already resolves a viewport, the adapter pairs it with the nested public gesture request as daemon-internal metadata. +ADR 0013 planning consumes that exact frame and does not probe the platform a second time. Target-relative swipes reuse the target-resolution observation. -Their typed single-pointer motion enters ADR 0013 as canonical `pan` with internal `endpoint-hold` metadata. +The resulting typed single-pointer motion enters ADR 0013 as the canonical `pan` input. +The `endpoint-hold` execution profile is carried as daemon-internal metadata. Maestro code cannot construct or execute two-pointer pan, pinch, rotate, transform, or physical pointer trajectories. -Simple successful target queries return match, visibility decision, candidate count, and observation generation in one response. -The daemon may retain the provider snapshot behind the evidence. -Geometry remains action-local. -Raw hierarchies, screenshots, and full candidate lists are failure/debug artifacts, not successful-path requirements. - -The daemon adapter may retain a successful observation's provider snapshot without exposing it through the engine contract. -A following target resolution may use it only as semantic evidence. -Maestro selectors use the shared structurally normalized provider snapshot before canonical iOS interactive presentation. -They intentionally do not request public `snapshot --raw`, which bypasses shared pruning and overlay annotations. -On iOS, the adapter derives canonical interactive presentation from the same snapshot without another capture. -Raw matching selects the node and authored index. -If the same source node has canonical interactive bounds, tap-like actions use them. - -A unique exact canonically hittable match may dispatch with its resolved point. -XCTest then binds live selector identity and coordinate delivery atomically. -A structured live-selector miss, ambiguity, point mismatch, or offscreen result returns to fresh Maestro resolution. +Simple successful target queries return their match, visibility decision, candidate count, and observation generation in one response. +The daemon may retain the provider snapshot behind that evidence, but geometry stays action-local. +Raw hierarchies, screenshots, and complete candidate lists are failure/debug artifacts, not happy-path requirements. + +The daemon adapter may retain the provider snapshot behind a successful observation without exposing it through the engine contract. +A following target resolution may use that snapshot only as semantic evidence. +Maestro selector semantics use the shared structurally normalized provider snapshot, before iOS canonical interactive presentation. +They deliberately do not request the public `snapshot --raw` shape, whose output contract bypasses shared pruning and overlay annotations. +On iOS, the adapter derives the provider's canonical interactive presentation from that same snapshot without another capture. +Raw matching still selects the node and authored index. +If that same source node has canonical interactive bounds, tap-like actions use those bounds. +A unique exact, canonically hittable match may be dispatched with its resolved point. +XCTest then binds the live selector identity and coordinate delivery atomically. +A structured live-selector miss, ambiguity, point mismatch, or off-screen result falls back to fresh Maestro resolution. All other targets capture fresh geometry before coordinate dispatch. Maestro visibility does not apply agent-device's React Native overlay blocking policy. Debug-overlay handling remains an explicit public-command workflow. -A scroll view or tab strip can remain in motion while visibility is true. -Thus, an observation frame never authorizes a later interaction, even within one mutation generation. -Before dispatch, every mutation attempt invalidates retained evidence. -This includes failed dispatch because the adapter cannot prove that the app remained unchanged. - -Maestro resolution preserves upstream first-match and explicit-index semantics. -Before authored index selection, equivalent iOS accessibility wrapper chains normalize to the semantic control or deepest matching leaf. -This matches XCTest element queries without changing distinct nested matches. -It does not adopt public agent-device unique-match requirements. -Conversely, nested public-command `AMBIGUOUS_MATCH` is an agent-device authoring failure. +Visibility can be true while a scroll view or tab strip is still in motion. +Thus, an observation frame is never authoritative for a later interaction, even within the same mutation generation. +Every mutating attempt invalidates retained evidence before dispatch. +This includes an attempt whose dispatch reports failure, because the adapter cannot prove that the app stayed unchanged. + +Maestro target resolution preserves upstream first-match and explicit-index semantics. +Before an authored index applies, equivalent iOS accessibility wrapper chains are normalized to the semantic control or the deepest matching leaf. +This matches XCTest element-query behavior and does not change distinct nested matches. +Maestro target resolution does not adopt the unique-match requirement of the public agent-device command surface. +Conversely, an `AMBIGUOUS_MATCH` from a nested public command is an agent-device authoring failure. Maestro `optional` does not suppress it. -Atomic iOS dispatch handles it through fresh Maestro resolution, as above. -Cancellation and infrastructure failures are also non-optional. +Atomic iOS dispatch handles that result with fresh Maestro resolution, as described above. +Cancellation and infrastructure failures are likewise non-optional. -Mutations that can leave transitions in flight record a pending stabilization boundary. -Before another mutation, the runtime samples at compatibility polling cadence until the hierarchy stabilizes. -It retains the final observation for the next read. +Mutations that can leave an in-flight transition record a pending stabilization boundary. +Before another mutating command, the runtime samples at the compatibility polling cadence until the hierarchy is stable. +The runtime retains that final observation for the next read. Commands with specialized completion semantics settle inline instead. -Hierarchy signatures project available semantic attributes, topology, and fixed-order integer edges. +Hierarchy signatures project the available semantic attributes, tree topology, and fixed-order integer edges. They exclude provider-only metadata, object-key order, and subpixel noise. -An authored observation after mutation skips the pending barrier and immediately polls its own condition. -Its successful snapshot becomes the boundary's baseline without discharging mutation order. +An authored observation after a mutation skips the pending barrier and polls its own condition immediately. +Its successful snapshot becomes the baseline for the boundary but does not discharge mutation ordering. If another mutation follows, the runtime waits one polling interval and compares a fresh hierarchy with that baseline. -If the UI still changes, it continues until stable. -If no later mutation follows, it performs no extra capture. -Upstream Maestro instead settles before assertion evaluation. -This intentional difference retains the previous engine's faster reads while preventing later tap, gesture, scroll, or input from overtaking transitions. +If the UI is still changing, the runtime continues until the hierarchy is stable. +If no later mutation follows, the runtime performs no extra capture. +This intentionally differs from upstream Maestro, which settles before it evaluates an assertion. +It retains the faster read behavior of the previous agent-device compatibility engine. +It does not allow a later tap, gesture, scroll, or input to overtake an in-flight transition. `retryTapIfNoChange` defaults to false. -If explicitly enabled, the runtime compares stable hierarchy with target-resolution hierarchy. -On iOS, unchanged hierarchy triggers Maestro's screenshot comparison. -It retries once only if both surfaces remain unchanged. -Screenshot evidence is best-effort because failed capture cannot prove safe mutation repetition. -The stable hierarchy prepares the next command without another hierarchy read. - -`waitForAnimationToEnd` owns a screenshot-stability operation. -It matches upstream's two immediate captures per attempt and 0.005% normalized absolute RGB-difference threshold. -These captures bypass ordinary screenshot stabilization to avoid a recursive wait on another stabilization policy. -After successful authored visual wait, screenshot stability discharges the same-generation pending mutation boundary. -No hierarchy capture precedes that visual barrier. -If no stable pair appears before the visual deadline, the boundary remains pending for the next mutation's hierarchy stabilization. -On iOS, comparison captures use the persistent runner's screenshot surface. -Both frames therefore use one initialized transport without simulator screenshot setup between polls. +If it is explicitly enabled, the runtime compares the stable hierarchy with the target-resolution hierarchy. +On iOS, an unchanged hierarchy triggers the same screenshot comparison that Maestro uses. +The runtime retries once, and only if both surfaces remain unchanged. +Screenshot evidence is best-effort because a failed capture cannot prove that the repetition of a mutation is safe. +The stable hierarchy primes the next command, so the retry policy does not add another hierarchy read there. + +`waitForAnimationToEnd` uses its own screenshot-stability operation. +It matches upstream's two immediate captures per attempt and the 0.005% normalized absolute RGB-difference threshold. +These captures explicitly bypass ordinary screenshot stabilization. +Thus, the command observes the application and does not recursively wait on another settling policy. +After a successful authored visual wait, screenshot stability discharges the same-generation pending mutation boundary. +No hierarchy capture precedes the visual barrier. +If the visual deadline arrives before a stable pair is observed, that boundary stays pending for the hierarchy settle of the next mutating command. +On iOS, comparison captures use the screenshot surface of the persistent runner. +Thus, both frames come from one warmed transport, and no simulator screenshot setup occurs between polls. Upstream Maestro is a version-pinned development reference, not a production dependency. -`scripts/maestro-conformance` enforces a three-layer oracle (issue #1274) instead of a manually typed parser fixture: - -1. Layer 1 runs pinned `maestro-orchestra` YAML parser `dev.mobile:maestro-orchestra:2.5.1` over vendored upstream flows and targeted failure-class flows. - Regeneration verifies its SHA. - The harness captures each parse. - A deterministic verifier replays captures against this engine. - Every flow becomes identical / both-reject / we-reject / mismatch. - Every non-identical outcome requires an explicit declared divergence. -2. Layer 2 reads upstream retry cap, animation-wait threshold/timeout, and erase cap directly from pinned bytecode. - It also reads parser-observed swipe-duration defaults. - It compares each value with live `MAESTRO_COMPATIBILITY_PRESETS`. -3. Layer 3 runs app-observable differential scenarios through both engines on the real fixture app. - These include stabilization order, tap retries, and optional warned-vs-failed outcomes. - It pins Maestro CLI to the same version as layers 1–2. - It runs on a schedule, never per PR. - -Both generated layers are committed and verified in CI through `node --test` without Java. -Per-PR CI cannot regenerate them. -Each fixture therefore carries a `contentHash` seal recomputed by the verifier. -Manual changes to captured commands or constants fail the build. -Scheduled `conformance-regenerate` reruns the harness against pinned jars and fails on any byte difference. -Thus, checks enforce upstream generation. - -Layer 3 uses the same declared-divergence contract as Layer 1. -A failing scenario declares `knownDivergence` with a required tracking issue. -Known gaps keep the schedule green; only undeclared divergences fail. -A declaration that no longer reproduces also fails. -Thus, a fix PR must remove it, and the differential becomes its acceptance test. -The measurement instrument must remain usable before its detected defects are repaired. +Conformance is enforced by the three-layer oracle in `scripts/maestro-conformance` (issue #1274). +That oracle replaced the original hand-typed parser fixture. + +1. Layer 1 drives the pinned `maestro-orchestra` YAML parser (`dev.mobile:maestro-orchestra:2.5.1`) over a corpus. + The corpus contains vendored upstream flows plus targeted bug-class flows. + The pinned parser is SHA-verified at regeneration. + Layer 1 captures each parse. + The deterministic verifier replays that capture against our engine. + The verifier classifies every flow as identical / both-reject / we-reject / mismatch. + Each non-identical outcome must be a declared, on-the-record divergence. +2. Layer 2 reads upstream constants (retry cap, animation-wait threshold/timeout, erase cap) straight from the pinned bytecode. + It also reads parser-observed defaults (swipe duration) straight from the pinned bytecode. + It cross-checks each value against the live `MAESTRO_COMPATIBILITY_PRESETS` constant. +3. Layer 3 runs a small set of app-observable differential scenarios through both engines against the real fixture app. + The scenarios cover settle ordering, tap retries, and optional warned-vs-failed. + Layer 3 pins the Maestro CLI to the same version as layers 1-2. + It is scheduled, never per-PR. + +Both generated layers are checked in and verified in CI via `node --test` with no Java. +Per-PR CI cannot re-derive them. +Each fixture therefore carries a `contentHash` seal that the verifier recomputes. +Thus, a hand edit to a captured command or constant fails the build. +The scheduled `conformance-regenerate` job then re-runs the harness against the pinned jars and fails on any byte difference. +That job makes "generated from upstream" enforced rather than documented. + +Layer 3 carries the same declared-divergence contract as layer 1. +A scenario that currently fails declares a `knownDivergence` with a required tracking issue. +The schedule stays green on that known gap, and only undeclared divergences fail. +A declaration that stops reproducing also fails. +Thus, the fix PR must remove the declaration, and the differential becomes the acceptance test for its own findings. +This contract exists because the instrument must not be blocked on the repair of what it just measured. Each of the four #1217 regressions has a dedicated fixture: - Percent rounding/rejection. - Target-swipe direction default. - Retry-cap semantics. -- Stabilization-loop order. +- Settle-loop ordering. -A pure unit test fixes percent truncation's runtime behavior because a one-pixel delta is not app-observable. -Failure class 4 uses an engine-side timing invariant on the device path, unit-tested with synthetic traces. -A declared `scrollUntilVisible` divergence (#1299) currently blocks it. -The differential itself detected that gap on its first working run. +A pure unit test, rather than a device scenario, pins the runtime half of percent truncation, because a one-pixel delta is not app-observable. +The detector for bug class 4 is an engine-side timing invariant that executes on the device path. +Unit tests exercise that detector against synthetic traces. +On the device path, the detector is currently blocked by a declared `scrollUntilVisible` divergence (#1299). +The differential itself found that divergence on its first working run. -Expected-divergence entries encode intentional deviations: +Documented intentional deviations are encoded as expected-divergence entries rather than silent mismatches: -- Hierarchy-vs-screenshot animation wait, with matching constants but different mechanisms. -- Horizontal-swipe in-page presets. -- Condition polls used for stabilization. +- The hierarchy-vs-screenshot animation wait (the constants match, but the mechanism differs). +- The horizontal-swipe in-page presets. +- Condition-poll-as-stabilization. - Ambiguity strictness under `optional`. -- Omitted iOS 3s pre-tap static gate, `IOSDriver.SCREEN_SETTLE_TIMEOUT_MS`, retained as a layer-2 reference-only vector. +- The omitted iOS 3s pre-tap static gate (`IOSDriver.SCREEN_SETTLE_TIMEOUT_MS`, a layer-2 reference-only vector). ## Performance Contract -Across Android/iOS pager and react-navigation corpora, the direct engine preserves: +The direct engine preserves these properties on Android and iOS across the pager and react-navigation corpora: -- Total wall time no slower than the pre-migration compatibility engine. -- Immediate authored condition polls after mutation, with that result as a later mutation's stabilization baseline. -- No second hierarchy capture solely to verify evidence produced within the same command. -- One direct viewport query and no accessibility capture for absolute coordinate swipes. -- Deferred mutation-to-mutation stabilization at the next command boundary, with its result reused by the following read. -- Exact authored endpoints after percentage swipe conversion. -- Helper/runner startup amortized across a suite. -- Separate reports for p50/p95 command latency, hierarchy/screenshot captures, tap retries, and transferred hierarchy bytes. -- Failure-only diagnostics excluded from successful-path latency comparisons. +- Total wall time is no slower than the pre-migration compatibility engine. +- An observation immediately after a mutation polls its authored condition directly. + A later mutation uses that result as its stabilization baseline. +- No command captures a second hierarchy only to re-verify evidence produced within that command. +- Absolute coordinate swipes perform one direct viewport query and no accessibility capture. +- Mutation-to-mutation stabilization is deferred to the next command boundary and primes the following read. +- Percentage swipe conversion preserves authored endpoints exactly. +- Helper/runner startup remains amortized across a suite. +- p50/p95 command latency, hierarchy and screenshot captures, tap retries, and transferred hierarchy bytes are reported separately. +- Failure-only diagnostics are excluded from happy-path latency comparisons. Android verification must prove the bundled helper backend and version. iOS verification must separate runner startup from warm command latency. -Both platforms rerun non-Maestro gesture canaries. +Both platforms rerun the non-Maestro gesture canaries. Their two-pointer plans, executor selection, and app-observable effects must remain unchanged. ## Consequences - agent-device supports selected Maestro Flow syntax and behavior. Unsupported features return explicit errors. - Conformance fixtures declare intentional differences. -- By decision #1292, `${...}` interpolation remains variable-lookup-only. - Upstream GraalJS expression evaluation is not duplicated outside `evalScript`. - Unresolved or expression-shaped payloads fail explicitly with source context. - Available computation paths are: - - `evalScript`: inline expression → `output.*` leaves. - - `runScript`: file computation → `${output.x}` consumption. - Only `evalScript` evaluates its payload as JavaScript. - Flow env and prior `output` leaves bind as strings in a `node:vm` context. - Assigned `output` becomes the flat string-key model. + Intentional differences are declared in the conformance fixtures. +- By decision (#1292), `${...}` interpolation stays variable-lookup-only. + Upstream's GraalJS expression evaluation is not reimplemented outside `evalScript`. + Thus, unresolved or expression-shaped payloads fail loudly with source context. + The escape hatches are `evalScript` (inline expression → `output.*` leaves) and `runScript` (file compute) → `${output.x}` (consume). + `evalScript` is the one command whose payload is evaluated as JavaScript. + Flow env and prior `output` leaves are bound as strings in a `node:vm` context. + The assigned `output` object folds back into the flat string-key model. Thus, `${output.uppercaseName}` and `${output.list.length}` resolve. - `assertTrue` (#1295) uses the same lookup-only subset: literal values and bare `${VAR}` lookups with a fixed string-truthiness table. + `assertTrue` (#1295) is scoped to that same lookup-only subset. + That subset is literal values and bare `${VAR}` lookups, evaluated with a pinned string-truthiness table. `node:vm` is explicitly not a security sandbox. - A bound context's prototype chain still reaches host `Object`/`Function`, so untrusted expressions can escape it. - Daemon-local `runScript` already has this trust assumption. - `evalScript` inherits it but is also reachable through remote HTTP flows. - `MaestroEngineOptions.trustedScripts` therefore gates it. - HTTP boundary `restrictRemoteHttpRequest` sets this option from `publicNetworkOnly`. - If false, `evalScript` is refused outright. -- Two production engines or runtime fallback between them are rejected because they duplicate semantic and performance ownership. -- Source provenance and runtime values remain typed through execution. -- Compatibility policy stays local; shared runtimes/backends retain device behavior. + A bound context's prototype chain still resolves to host `Object`/`Function`, so an untrusted expression can escape it. + `runScript` already carried this trust assumption for daemon-local flows. + `evalScript` inherits the same assumption. + However, `evalScript` is reachable from a flow accepted over the daemon's remote HTTP surface. + Thus, `MaestroEngineOptions.trustedScripts` gates `evalScript`. + The HTTP boundary (`restrictRemoteHttpRequest`) sets that option from `publicNetworkOnly`. + If the option is false, `evalScript` is refused outright. +- Shipping two production engines or a runtime fallback between them is rejected because it doubles semantic and performance ownership. +- Source provenance and runtime values stay typed through execution. +- Compatibility policy remains local, while device behavior stays in shared runtimes and backends. - Cross-platform correctness may require richer provider query evidence, but not additional round trips. -- ADR 0013 internals can change while its normalized single-pointer boundary and executor guarantees remain available to Maestro. +- ADR 0013 can be rewritten internally without a change to Maestro, as long as its normalized single-pointer boundary and executor guarantees remain available. ## Alternatives Considered -- **Embed upstream Orchestra.** Rejected because Java startup, package weight, driver ownership, and platform coverage would remove performance and backend advantages. +- **Embed upstream Orchestra.** Rejected because Java startup, package weight, driver ownership, and platform coverage would erase performance and backend advantages. - **Build a shared replay VM first.** Rejected until native `.ad` needs structured runtime control flow. One caller does not justify a broader abstraction. -- **Continue compilation to typed `SessionAction` variants.** Rejected because it retains recursive replay dispatch and prevents direct ownership of observation generations. +- **Keep compiling to typed `SessionAction` variants.** Rejected because it retains the replay trampoline and prevents the compatibility engine from owning observation generations directly. From fa58ed38d7eea9133c84e67d53b37fa9723cd291 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 45/55] docs(adr): restore the frozen meaning in ADR 0016 Restore the #1533 invariant that no lifecycle state name, transition, or entry point is added or altered, and the writer's single publication-blocked question. Restore "only" for PUBLISHED in the body and glance, plus the "shipped" status of each amendment and the implemented-on-main status. Put superseded behavior (pre-amendment close paths, the recordSession flag) back in the past tense, and keep "such as" non-exhaustive. Restore dropped qualifiers: "own recorded evidence", "re-serialize", "for the matched element", "(2026-08-02 amendment)", "leaks ... exactly like", "entirely under #1336". --- .../0016-active-session-script-publication.md | 530 ++++++++++-------- 1 file changed, 288 insertions(+), 242 deletions(-) diff --git a/docs/adr/0016-active-session-script-publication.md b/docs/adr/0016-active-session-script-publication.md index 41d7853a51..e45326f958 100644 --- a/docs/adr/0016-active-session-script-publication.md +++ b/docs/adr/0016-active-session-script-publication.md @@ -3,58 +3,62 @@ ## Status Accepted. - -Completed process history is in git history. +This ADR is implemented on `main`, including the #1349 identity-level destination-guard amendment. Implementation owners are `packages/daemon/src/sessions/handlers/session-script-publication.ts` and `packages/daemon/src/core/session-script-active-publication.ts`. ## Rules at a glance -[Decision](#decision) contains binding contracts and refusal cases. - -- `session save-script [path] [--force]` publishes ordinary script recording without closing the app or session. - The session remains live at the destination. -- Only a new session's initial `open --save-script` establishes **ARMED**. - A second successful plain `open` causes **ABORTED**. - Successful save causes **PUBLISHED**. - ABORTED and PUBLISHED remain terminal until session destruction. - Filesystem/target failure leaves ARMED retryable. -- Publication requires exactly one initial recorded `open`, no unresolved session-local `@ref`, and a portable selector-`wait` destination guard after the last descriptor-classified mutation. - The guard's `target-v1` annotation must have `verification: "verified"` (#1349). - Every refusal precedes filesystem work and names recovery. -- A session with `saveScriptBoundary` refuses this action without repair-state changes (ADR 0012). -- The daemon session store supplies `ReplayCommandResult.sessionActive`. - Clients use this signal to retain the daemon after a close-less handoff. -- Sensitive `fill` values must use `fill --record-as ` (ADR 0017, #1348). - Unparameterized `fill`/`type` values persist literally, so an entered secret without `--record-as` is published. - Protection applies across the recording session (#1398). - Later actions cannot serialize app-rendered echoes of already-parameterized values. +This section is a normative summary. +[Decision](#decision) contains the binding contracts and refusal cases. + +- `session save-script [path] [--force]` publishes the current ordinary script recording without closing the app or the session. + The session stays live at the destination. +- Recording lifecycle: + - Only the initial `open --save-script` of a new session establishes **ARMED**. + - From ARMED, a second successful plain `open` causes **ABORTED**. + - From ARMED, only a successful atomic `session save-script` establishes **PUBLISHED**. + - ABORTED and PUBLISHED are terminal until the session is destroyed. + - A filesystem or target failure leaves the recording ARMED and retryable. +- Publication requires exactly one initial recorded `open`, a portable selector-`wait` destination guard after the last descriptor-classified mutating action, and no unresolved session-local `@ref`. + Since #1349, the `target-v1` annotation of the guard must be `verification: "verified"`. + Every refusal happens before filesystem work and names its recovery. +- Repair transactions (ADR 0012) are disjoint from this lifecycle. + A session with `saveScriptBoundary` set refuses this action without a change to repair state. +- `ReplayCommandResult.sessionActive` comes from the daemon's session store, never from a re-parse of the script. + This signal lets a one-shot client keep the daemon alive on a close-less handoff. +- Sensitive `fill` inputs must be recorded as placeholders through `fill --record-as ` (ADR 0017, shipped for #1348). + Unparameterized `fill`/`type` values persist literally into the artifact, so a secret entered without `--record-as` is published. + The protection is scoped to the recording session (ADR 0017's #1398 amendment). + A later action's own recorded evidence can never re-serialize an app-rendered echo of an already-parameterized value either. ## Context -The replay repair study found complete re-records cheaper than repairs in two of three measured drift classes. +The replay repair study found that re-recording a drifted journey was cheaper than repairing it in two of the three measured drift classes. The immediate reusable unit is an **open-to-destination script**. -This self-contained `.ad` script opens an app, reaches screen X, verifies a destination landmark, and leaves the session active for further work. +This self-contained `.ad` script opens the app, performs the complete journey to screen X, and verifies a destination landmark. +It leaves the app session active there, so an agent can continue with new work. -Ordinary recording combines two concerns. +Ordinary script recording currently combines two concerns. `open --save-script[=]` arms recording before the first interaction. -It captures selector chains and ADR 0012 `target-v1` identity evidence at action time. -`close` publishes accumulated actions, adds terminal `close`, and tears down the session. -That artifact serves closed test flows, not live starting states. - -Late publication from an ordinary unarmed session is insufficient. -History retains commands but intentionally discards the resolved target tree after each action. -Late publication can serialize selector text but cannot reconstruct the actual target identity. -Unconditional identity capture would change normal execution. +Thus, selector chains and ADR 0012 `target-v1` identity evidence are captured at action time. +`close` publishes the accumulated script, records a terminal `close`, and tears down the session. +That artifact is suitable as a closed test flow, but not as a live starting state. + +Publication only at the end of an ordinary unarmed session is insufficient. +Session history retains the commands. +But the resolved target tree that `target-v1` evidence needs is deliberately discarded after each action. +Late publication can serialize selector text, but it cannot reconstruct which element the action actually targeted. +Unconditional identity capture for every session would change normal interaction execution. Recording disables direct selector fast paths when evidence requires a capture-backed route. -This performance and data-retention change remains unmeasured. +That performance and data-retention change has not been measured. -Issue [#1346](https://github.com/callstack/agent-device/issues/1346) and the disposable [`session-save-script` prototype](https://github.com/callstack/agent-device/tree/agent/prototype-session-save-replay/scripts/prototypes/session-save-script) record the workflow and API trial. +Issue [#1346](https://github.com/callstack/agent-device/issues/1346) and the disposable [`session-save-script` prototype](https://github.com/callstack/agent-device/tree/agent/prototype-session-save-replay/scripts/prototypes/session-save-script) record the motivating workflow and API trial. Composable lifecycle-free fragments remain a separate decision in [#1336](https://github.com/callstack/agent-device/issues/1336). ## Decision -Add explicit publication for an already-armed ordinary recording: +Add an explicit publication action for an already-armed ordinary script recording: ```sh agent-device open com.example.app --relaunch --save-script=screen-x.ad @@ -63,88 +67,118 @@ agent-device wait 'role="heading" label="Screen X"' agent-device session save-script ``` -`session save-script [path] [--force]` publishes without app close or session deletion. -An explicit `path` retargets recording under existing target/force authorization rules. -Without it, publication uses the path from `open --save-script` or the existing generated default. +`session save-script [path] [--force]` publishes the current ordinary script recording without closing the app or deleting the session. +An explicit `path` retargets the recording under the existing target/force authorization rules. +Without it, publication uses the path armed by `open --save-script` or the existing generated default. ### Recording lifecycle -An eligible ordinary recording has three states: +An ordinary recording eligible for active-session publication has three states: -- **ARMED**: only a new session's initial successful `open --save-script[=]`, recorded as action zero, establishes this state. - The session records portable inputs and fresh target identity evidence. +- **ARMED**: only the initial successful `open --save-script[=]` of a new session, recorded as action zero, establishes this state. + The session records portable action inputs and fresh target identity evidence. - **ABORTED**: another successful plain `open` while ARMED establishes this state. The app operation may continue. - The recording no longer represents a single-open bootstrap, and close-time publication is disarmed. - The successful `open` warns that another script requires a fresh session. -- **PUBLISHED**: successful atomic `session save-script` commits complete history from the sole recorded `open` through the current action. - The session remains active at the destination, with close-time publication disarmed. - -Filesystem or target-collision failure preserves ARMED, path, and same-target `--force` authorization. -The caller can correct target or permissions and retry. -ABORTED and PUBLISHED remain terminal until session destruction. -In either state, `session save-script` fails explicitly without a write. - -Every arming entry point respects terminality. -On any existing session, `open --save-script` rejects before app dispatch. + But the recording is no longer a single-open bootstrap, and close-time publication is disarmed. + The successful `open` response warns that a fresh session is required to author another script. +- **PUBLISHED**: only a successful atomic `session save-script` establishes this state. + That command commits the complete history from the sole recorded `open` through the current action. + The session remains active at the destination, but close-time script publication is disarmed. + +A filesystem or target-collision failure leaves the recording ARMED, including its path and same-target `--force` authorization. +Thus, the caller can correct the target or permissions and retry. +ABORTED and PUBLISHED are terminal until the session is destroyed. +In either state, `session save-script` fails with an explicit error and never writes. + +Every existing arming entry point respects terminality. +On any existing session, `open --save-script` is rejected before app dispatch. This includes unarmed, ARMED, ABORTED, and PUBLISHED sessions. -A later plain `open` is permitted. -It changes ARMED to ABORTED and preserves ABORTED/PUBLISHED. -In ABORTED or PUBLISHED, `close --save-script[=]` rejects before platform close or filesystem work. -The caller can retry plain `close`. -Plain `close` tears down those states without a write. -Close of unpublished ARMED recording retains existing close-time publication. -Only a fresh session permits re-arming. - -**Amendment (2026-08-02).** A never-armed session has no fourth pre-ARMED state name here. -Late close-time arming can produce selector fallback chains without recording-time `target-v1` evidence. -Live evidence showed that callers received no signal that identity capture never ran. -Degraded replay verification is worse than explicit refusal. -Thus, `close --save-script` on never-armed sessions rejects before teardown or filesystem work. -It names `open --save-script` as recovery. -Plain `close` still tears down without a write. -This differs from [#1533](https://github.com/callstack/agent-device/issues/1533), which concerns an ARMED-then-ABORTED session. - -**Amendment (#1533).** A mutable `recordSession` evidence flag could disagree with ABORTED lifecycle state. -A later `--save-script` could enable evidence capture while status remained ABORTED. -A bare `close` could then publish through a writer that refused only repair transactions. -This contradicted the promised plain-close behavior after `close --save-script` refusal. -Clearing the flag in `abortAuthoringOnSecondOpen` depended on operation order instead of state construction. - -**Amendment (#1533 follow-up).** Recording is **derived**, not stored. -Routing flag writes through one owner still allowed a second source of truth to disagree with lifecycle state. -The stored `SessionState.recordSession` field is removed. -`isRecordingPublication` derives evidence capture from lifecycle: +A later plain `open` is allowed. +It changes ARMED to ABORTED and leaves ABORTED/PUBLISHED unchanged. +In ABORTED or PUBLISHED, `close --save-script[=]` is rejected before platform close or filesystem work. +Thus, the caller can retry with plain `close`. +Plain `close` tears down ABORTED/PUBLISHED without writing. +The close of an unpublished ARMED recording retains the existing close-time publication behavior. +A fresh session is the only re-arming boundary. + +**Amendment (2026-08-02, shipped).** A never-armed session has no recorded `open --save-script` at all. +The lifecycle above has no fourth pre-ARMED state name for it. +Before this amendment, such a session used the same close-time write path as an ARMED recording. +`close --save-script` on it added the request to the authoring lifecycle at record time and published anyway. +Live evidence showed that this produced a script whose actions had selector fallback chains but no recording-time `target-v1` evidence. +The caller got no signal that evidence capture never ran. +Degraded replay verification is worse than an explicit refusal. +Now, `close --save-script` on a never-armed session is rejected before any teardown or filesystem work, the same way as in ABORTED/PUBLISHED. +The refusal names `open --save-script` as the recovery. +A plain `close` still tears the session down without writing. +This case is distinct from [#1533](https://github.com/callstack/agent-device/issues/1533), which concerns an already-ARMED-then-ABORTED session. +In that session, flag ingress re-enabled recording and let a *bare* `close` publish. +A bare `close` has no `--save-script` on the close itself. +The amendments below resolve that case. + +**Amendment (#1533, shipped).** Before this amendment, only `abortAuthoringOnSecondOpen` enforced the ABORTED terminality above. +It cleared `session.recordSession`. +That enforcement depended on operation order, not on construction. +`recordSession` was an evidence-capture flag that several surfaces set directly. +Thus, a later `--save-script` re-armed it while the status stayed ABORTED. +A *bare* `close` then published the full session log through a writer that could refuse only repair transactions. +For `close --save-script` in ABORTED, this ADR specifies a refusal that promises the caller that plain `close` "tears down without writing". +Thus, the gap also made an existing error message untrue. + +ABORTED is now terminal by construction. +This amendment routed `--save-script` arming through one rule that the publication projection owned. +That rule answered "not recording" for an ABORTED lifecycle on every surface that handled the flag. +These surfaces were the re-open builder, the close finalizer, and the recorded-action ingress. +Thus, the flag could no longer contradict the status. +The follow-up amendment below replaces the stored flag with a derivation. +The aggregate likewise answers publication authorization. +The writer asks one publication-blocked question. +That question covers not-recording, an uncommittable or committed repair, and an ABORTED authoring lifecycle alike. +Thus, every path that reaches the writer refuses: bare `close`, teardown, idle-reap, and active publication. +ARMED and PUBLISHED lifecycles and every repair transaction are unchanged. +No state name, transition, or entry point in the lifecycle above is added or altered. + +**Amendment (#1533 follow-up, shipped).** The rule above still relied on a stored `SessionState.recordSession` field. +Every writer had to set that field in step with the lifecycle. +Routing the writes through one place made the two agree, but it did not make disagreement unrepresentable. +The field remained a second source of truth for a question that the aggregate already answered. + +Recording is now **derived**, not stored. +`isRecordingPublication` reads it from the lifecycle, and the field is gone: - Ordinary authoring records only while ARMED. -- Repair transactions record throughout their lifetime, including terminal statuses. - -Consequences of this derivation are: - -- No surface can arm recording without a lifecycle transition that authorizes it. - `buildNextOpenSession` and the close finalizer make no recording decision. -- The aggregate answers the writer's publication gate. - Every write path uses that gate, including bare `close`, teardown, idle-reap, and active publication. - A separate ABORTED check is unnecessary because terminal authoring already means no recording. -- Evidence capture and publication authorization share the aggregate but remain **distinct predicates**. - For ordinary authoring, ARMED records and publishes; ABORTED/PUBLISHED do neither. - For repair, `isRecordingPublication` remains true even for `committed` and `aborted`. - Publication additionally requires `isRepairArmedWriteBlocked` to permit the write. - It refuses committed transactions as idempotent no-ops and transactions not yet committable (ADR 0012 decision 6, C2). - -A repair that captures evidence cannot necessarily publish. -Combining these predicates could republish committed transactions or commit an incomplete prefix. -The derivation preserves the former flag's behavior at every transition. -Whether committed repairs should still capture recording-time evidence remains unanswered. -The former flag permitted it. -Changing it would change behavior, not merely derive state. - -This lifecycle is distinct from ADR 0012 repair transactions. -If `saveScriptBoundary` is set, `session save-script` rejects. -Recovery uses the existing `replay --from` and teardown commit protocol to finish or abort repair. -Active publication never: - -- Marks repair COMPLETE. +- A repair transaction records for its whole lifetime, terminal statuses included. + +The derivation has three consequences: + +- No surface can arm recording without a move of the lifecycle that authorizes it. + Thus, the #1533 drift is unrepresentable, not only guarded against. + `buildNextOpenSession` and the close finalizer now make no recording decision at all. +- The aggregate answers the whole publication gate of the writer. + The separate ABORTED check of the writer is gone, because a terminal authoring lifecycle is already not recording. + Thus, no second gate can disagree with the first about *authoring*. +- Evidence capture and publication authorization now derive from the same aggregate. + But they deliberately remain **distinct predicates**. + - They coincide for ordinary authoring: ARMED both records and publishes, and ABORTED and PUBLISHED do neither. + - They do NOT coincide for repair. + `isRecordingPublication` is true for every repair status, `committed` and `aborted` included. + Publication additionally requires `isRepairArmedWriteBlocked` to pass. + That check refuses a committed transaction (idempotent no-op) and a transaction that is not yet committable (ADR 0012 decision 6, C2). + - Thus, a repair that captures evidence is not necessarily a repair that may publish. + A merge of the two predicates would silently republish or commit a prefix. + +This change is behavior-preserving. +The derivation reproduces exactly what the flag held at every transition. +This amendment deliberately does not answer one real question: should a *committed* repair still capture recording-time evidence? +The old flag said yes. +A change to that answer is a behavior change, not a derivation. + +This lifecycle is distinct from the ADR 0012 repair transaction. +`session save-script` rejects a session with `saveScriptBoundary` set. +The refusal directs the caller to finish or abort the repair through its existing `replay --from` and teardown commit protocol. +Active-session publication never: + +- Marks a repair COMPLETE. - Commits a healed slice. - Writes `# agent-device:heal-complete`. - Changes repair tombstone semantics. @@ -152,180 +186,192 @@ Active publication never: ### Destination readiness and replay handoff The destination is an authored postcondition, not the last navigation action. -After the last mutating action, the recorded suffix must contain a portable selector or selector-chain `wait` for a ready-screen landmark. -Duration waits, `wait stable`, and `wait @ref` do not qualify. -`wait stable` may follow the landmark wait. -Before filesystem work, the publisher validates the serialized guard. -It never relies on repair-only bare-ref rejection. -Without this **destination guard**, `session save-script` refuses and tells the author to record one. -V1 neither infers screen identity from snapshots nor synthesizes implicit guards. - -**Amendment (#1349).** The guard requires identity, not only a selector. -A selector wait recorded while ARMED captures landmark-mode `target-v1` evidence (ADR 0012's read-only-step amendment). -Only a selector wait with `verification: "verified"` qualifies. -An identity-empty element without id or label records no annotation and cannot qualify. -Publication refuses it with a recovery hint for a labeled/id-bearing landmark. - -Replay retains polling until a selector match has the recorded local identity and leaf-anchored ancestry prefix. -If only same-selector impostors exist at deadline, it fails closed with `identity-mismatch` `REPLAY_DIVERGENCE` before wait success. -A provider regression records, publishes, then replays against a rearranged tree whose same-label node has different id/ancestry. -It proves rejection of that false success. - -**Amendment (#1398, ADR 0017).** Echo-only identity also cannot qualify. -If a selector wait's only identity echoes a literal already parameterized through `fill --record-as`, it records no annotation. -Publication uses the same refusal hint and directs the author to a stable non-value-bearing landmark. -This enforces the manual correction in #1398's motivating scenario. - -The original selector-only guard proved existence, not landmark identity. -A rearranged screen could pass with the same weak label elsewhere. -[#1349](https://github.com/callstack/agent-device/issues/1349) owns identity design for waits and remaining read-only steps. -ADR 0012's pre-action verification cannot apply unchanged because an absent landmark is an expected initial wait condition. -The identity check therefore runs after selector resolution and before wait success. -ADR 0012's #1349 amendment defines the mechanism and read-only coverage classification. - -A request-sensitive recording-effect trait on central `CommandDescriptor` defines the last mutating action. -Every command that records session actions requires this trait under a completeness test. -The publisher has no local command-name set. -For `find`, `keyboard`, and `alert`, the trait distinguishes read-only from mutating subcommands. -Conservative `refFrameEffect: may-invalidate` is insufficiently precise here. - -Consumption preserves existing close-less replay behavior. -The named session remains active, and successful `ReplayCommandResult` returns its `session` id. -The caller binds subsequent commands to that id. -Replay reports success only after guard completion. -Absence of `close` changes neither action dispatch nor success timing. - -**Response shape (issue #1384).** `ReplayCommandResult` requires `sessionActive: boolean`. -Response construction computes it from the daemon store's current existence of `session`, never from script parse. -This enables the real CLI/IPC client to retain active handoff sessions. -Without it, one-shot `replay`/`test` teardown could not distinguish live handoff from completed work and stopped the owning daemon. -The client owner is `src/daemon-client/daemon-client-lifecycle.ts`. -`sessionActive` is true for every close-less run, including `--from` resume. -It becomes false after terminal `close` or the deferred equivalent in repair-armed runs. +Before publication, the recorded suffix after the last mutating action must contain a `wait` whose target is a portable selector or selector chain. +That selector must identify a landmark on the ready destination screen. +A duration wait, `wait stable`, or `wait @ref` does not qualify. +But `wait stable` may follow the landmark wait. +The publisher validates the serialized guard before filesystem work. +The publisher never relies on repair-only bare-ref rejection. +Without this **destination guard**, `session save-script` refuses publication and tells the author to record one. +V1 does not infer a screen identity from a snapshot or synthesize an implicit guard. + +**Amendment (#1349, shipped).** The guard is now identity-level, not merely selector-level. +A selector wait recorded while ARMED captures landmark-mode `target-v1` evidence for the matched element (ADR 0012's read-only-step amendment). +A qualifying destination guard is exactly a selector wait whose annotation is `verification: "verified"`. +A selector wait on an identity-empty element (no id and no label) records no annotation and does not qualify. +Publication refuses it with a recovery hint that names a labeled/id-bearing landmark. + +On replay, polling is preserved. +The wait keeps polling until a selector match carries the recorded identity (local identity + leaf-anchored ancestry prefix). +If the wait reaches its deadline with only same-selector impostors, it fails closed as an `identity-mismatch` `REPLAY_DIVERGENCE` before the wait reports success. +A provider-scenario regression covers the reshuffled-screen false-pass below. +That regression records, publishes, then replays against a reshuffled tree whose same-label node is under a different id/ancestry. + +**Amendment (#1398, ADR 0017).** An identity-empty landmark is not the only case that fails to qualify. +A selector wait can have an app-rendered echo as its only identity. +If that echo is of a literal that the recording session already parameterized through `fill --record-as`, the wait also records no annotation. +This is ADR 0017's session-scoped echo protection. +Thus, that wait does not qualify as a destination guard either. +Publication refuses it with the same recovery hint and directs the author to a stable, non-value-bearing landmark. +This refusal enforces exactly the fix that the motivating scenario of issue #1398 applied by hand. + +The original selector-level caveat is retained as context. +The guard proved that an element that matches its selector exists. +It did not prove that the element is the same landmark element observed while authoring. +Thus, a reshuffled screen with the same weak label elsewhere could false-pass. +[#1349](https://github.com/callstack/agent-device/issues/1349) owned the identity design for waits and remaining read-only steps. +That design preserves polling. +ADR 0012's pre-action `target-v1` verification cannot be attached to a wait unchanged, because a not-yet-present landmark is the expected starting condition. +Thus, the identity check runs after the wait's selector resolves and before the wait reports success. +ADR 0012's #1349 amendment defines the mechanism and the read-only-step coverage classification. + +The phrase "last mutating action" derives from a request-sensitive recording-effect trait on the central `CommandDescriptor`. +Every command that records session actions requires this trait, and a completeness test guards it. +It is not a publisher-local command-name set. +The trait distinguishes app-state mutation from observation for subcommands, such as read-only versus mutating `find`, `keyboard`, and `alert` actions. +The existing conservative `refFrameEffect: may-invalidate` classification is not precise enough for this boundary. + +On consumption, a script without `close` preserves the existing replay behavior. +The named session stays active, and the successful `ReplayCommandResult` returns its `session` id. +The caller binds subsequent commands to that returned id. +Replay reports success only after the destination guard completes. +The absence of `close` changes neither action dispatch nor when replay reports success. + +**Response shape (issue #1384).** `ReplayCommandResult` carries a required `sessionActive: boolean`. +The daemon computes it when it builds the response, from whether `session` still exists in the daemon's own store. +The daemon never computes it by a re-parse of the script for `close`. +This field lets the real CLI/IPC client (not only the in-process handler) actually honor "the session stays active". +Before this field, the response had no explicit signal. +The one-shot `replay`/`test` teardown of the client had no way to distinguish a still-active handoff from a finished one. +It tore the owning daemon down regardless (`src/daemon-client/daemon-client-lifecycle.ts`). +`sessionActive` is `true` for every close-less run, including a `--from` resume. +It is `false` once the terminal `close` of the script, or the deferred equivalent of a repair-armed run, has executed. ### Sensitive inputs Executable `.ad` artifacts serialize ordinary action inputs literally. -Diagnostic/event-log redaction cannot protect inputs that replay must execute. +Diagnostic and event-log redaction cannot protect an input that replay must later execute. ADR 0017 resolves #1348 for explicit sensitive fills. -`fill ... --record-as PASSWORD` sends live text to the app but records only `${PASSWORD}`. +`fill ... --record-as PASSWORD` sends the live text to the app while it records only `${PASSWORD}`. Replay receives the value through `AD_VAR_PASSWORD` or `--env PASSWORD=`. -This behavior is opt-in and fill-only. +This protection is opt-in and fill-only. Unparameterized fill/type inputs remain literal artifact content. -Authors must use ADR 0017 for each sensitive fill and avoid secret-bearing `type` steps. -CLI help places the safe workflow and literal-input warning beside publication guidance. +Thus, authors must use ADR 0017 for each sensitive fill and avoid secret-bearing `type` steps. +CLI help states both the safe workflow and the remaining literal-input warning next to publication guidance. -ADR 0017's #1398 amendment protects the whole recording session. -Later unrelated actions cannot serialize an already-parameterized literal's app-rendered echo in their result or `target-v1` evidence. -A value protected once remains protected for all subsequent recorded actions in that session. +ADR 0017's #1398 amendment extends this protection to the whole recording session. +A later, unrelated recorded action can no longer re-serialize an app-rendered echo of an already-parameterized literal into its own result or `target-v1` evidence. +Thus, a value protected once by `--record-as` stays protected for every action recorded after it in the same session, not only in its own originating fill. ### Artifact contract The published `.ad`: -- Contains exactly one initial recorded `open` and every recordable action through publication request. -- Contains a portable selector/selector-chain destination guard after its last descriptor-classified mutation. -- Appends neither `session save-script` nor `close`. -- Uses ordinary session context header, selector-chain optimization, and canonical `target-v1` annotations captured while ARMED. -- Fails explicitly on unresolved session-local `@ref` or missing target evidence required by ADR 0012. +- Contains exactly one recorded `open` as its first action, and every recordable action through the publication request. +- Contains a portable selector/selector-chain destination guard after its last descriptor-classified mutating action. +- Does not append or serialize `session save-script` or `close`. +- Uses the ordinary session context header, selector-chain optimization, and canonical `target-v1` annotations captured while ARMED. +- Fails with an explicit error instead of emitting an unresolved session-local `@ref`. + It also fails with an explicit error instead of dropping target evidence that ADR 0012 requires for an element-targeting recorded action. - Uses the existing same-directory atomic publication primitive. - Every existing target is refused unless `--force` authorizes atomic replacement. + It refuses every existing target unless `--force` authorizes atomic replacement. -Success identifies final path and session and reports serialized action count. -Before any write, the command must fail if: +The success response identifies the final path and session and reports the number of serialized actions. +The command must fail before writing in each of these cases: -- No active session exists. +- There is no active session. - Recording was not armed. -- Recording is ABORTED or PUBLISHED. -- History lacks exactly one initial `open`. +- The recording is ABORTED or PUBLISHED. +- The history does not contain exactly one initial `open`. - No portable destination guard exists. - A repair transaction owns the session. -Every failure explains recovery. -None becomes `{ written: false }` success. +Every failure explains the recovery action. +No failure degrades to `{ written: false }` success. ### Surface and naming -V1 extends existing `session` and the typed session client surface. -It adds no `script start/stop`, marks, or second replay engine. -CLI help makes both phases explicit: +V1 extends the existing `session` command and the typed session client surface. +It does not introduce `script start/stop`, marks, or a second replay engine. +CLI help makes the two phases explicit: -1. `--save-script` arms evidence capture. +1. The existing `--save-script` flag arms evidence capture. 2. `session save-script` publishes without teardown. -This ADR neither renames nor deprecates `--save-script`. -The flag and session action identify the same artifact. -`open --save-script` configures the armed recording's eventual target. -`session save-script` requests immediate publication. -`save-replay` is rejected because replay executes the script instead of naming the saved artifact. +This ADR does not rename or deprecate `--save-script`. +The flag and the session action name the same persisted artifact. +`open --save-script` configures the eventual target of the armed recording. +`session save-script` requests publication now. +`save-replay` is rejected, because replay is the act of executing that script, not the artifact being saved. ## Consequences -- Agents can record onboarding or deep navigation as a self-contained starting state, replay it from the start, and continue in the live session. -- The workflow needs two explicit phases. - Evidence capture must start before the first target action. - The destination becomes known only at publication. -- Ordinary unarmed interactions retain fast paths and retention behavior. -- Successful active publication cannot conflict with later close-time auto-save. -- A second successful open abandons the current artifact instead of silently publishing a multi-open bootstrap. - Only a fresh session resumes authoring. -- Intermediate lifecycle-free fragments, entry guards, includes, composed digests, and shared fragment pins remain under #1336. -- Sensitive fills use ADR 0017 `--record-as `. +- Agents can record onboarding or deep navigation as one self-contained starting state, replay it from scratch, and continue from the resulting live session. +- The workflow has two explicit moments. + Evidence must be armed before the first target action. + The destination is known only when the caller publishes. +- Normal unarmed interactions keep their current fast paths and retention behavior. +- A successful active-session publication cannot collide with a later close-time auto-save. +- A second successful open abandons the in-flight artifact instead of silently publishing a multi-open bootstrap. + Authoring resumes only in a fresh session. +- Intermediate lifecycle-free fragments, entry guards, include semantics, composed digests, and shared fragment pinning remain entirely under #1336. +- Secret-bearing fill authoring uses ADR 0017's explicit `--record-as ` contract. Unparameterized fill/type inputs remain literal script content. - Arbitrary history ranges remain outside scope. -- Close-less `replay` leaves a live daemon and app session (#1384). - This includes the ordinary one-shot invocation's owned/ephemeral daemon. - The client retains it and reports a `--state-dir` address hint. - Unattended close-less replay therefore retains a session like interactive open. - Ordinary idle reap or explicit `close` bounds its lifetime, not the one-shot process. - This intentionally assigns close ownership to the caller. - Recorded test flows already end with `close`, so `test` runs remain unaffected. + Arbitrary history ranges remain out of scope. +- On consumption (issue #1384), a `replay` whose script has no terminal `close` leaves a live daemon and app session behind. + This includes the owned/ephemeral daemon of the ordinary one-shot `replay` invocation. + The client keeps that daemon alive and reports a `--state-dir` address hint for it instead of a teardown. + Thus, an unattended close-less replay (for example, in CI) leaks a session exactly like a session opened interactively. + Ordinary idle-reap or an explicit `close` bounds it, never the process lifetime of the one-shot command. + This is the intended shape of "the caller owns close", not an accident. + Recorded test flows already end with `close`, so `test` runs are unaffected. ## Alternatives Considered -- **Save any session history at the end.** Rejected because target identity cannot be reconstructed after interaction. - The artifact would violate ADR 0012 provenance. -- **Capture full target evidence in every session.** Rejected until direct-path latency, capture count, memory, and event-log costs are measured. - It would change ordinary interaction behavior merely to shorten one authoring command. -- **`close --no-close` or `--save-script --no-close`.** Rejected because a teardown-named command would conditionally retain sessions. +- **Save any session history at the end.** Rejected, because target identity evidence cannot be reconstructed after the interaction. + The resulting artifact would weaken ADR 0012's provenance model. +- **Capture full target evidence in every session.** Rejected until its direct-path latency, capture count, memory, and event-log costs are measured. + It would alter ordinary interaction behavior to make one authoring command shorter. +- **`close --no-close` or `--save-script --no-close`.** Rejected, because a command named for teardown would conditionally preserve the session. It also would not solve late arming. -- **General `script start/stop` or history marks.** Rejected because v1 covers exactly one recorded `open` through one destination. - Arbitrary slices need entry-state semantics and belong to fragment design. -- **`replay save`.** Rejected because `replay ` consumes artifacts, while publication consumes live sessions. - The session owns source data and lifecycle. -- **Infer destination fingerprints at publication.** Rejected for v1 because screen identity and readiness are app semantics. - Caller-authored waits are explicit, recordable, and fail at the correct point in cold replay. +- **General `script start/stop` or history marks.** Rejected, because the accepted v1 boundary is exactly one recorded `open` through one destination. + Arbitrary slices require entry-state semantics and belong with fragment design. +- **`replay save`.** Rejected, because `replay ` consumes an artifact while publication consumes a live session. + The session owns the source data and lifecycle. +- **Infer a destination fingerprint at publication.** Rejected for v1, because screen identity and readiness are app semantics. + A caller-authored target wait is explicit, already recordable, and fails at the correct point during cold replay. ## Validation Required for Implementation -- Unarmed sessions refuse publication before filesystem work and name `open --save-script` as recovery. -- Armed sessions without destination guards refuse before filesystem work and name selector-targeted `wait` as recovery. - Tests include `wait @ref`, duration waits, and `wait stable` refusals. +- An unarmed session refuses publication before filesystem work and names `open --save-script` as the recovery. +- An armed session without a destination guard refuses publication before filesystem work and names a selector-targeted `wait` as the recovery. + `wait @ref`, duration waits, and `wait stable` are covered refusal cases. - A second successful plain `open` changes ARMED to ABORTED and disables all publication. - Existing-session `open --save-script` rejects before app dispatch. -- Armed sessions publish `open` and annotated actions without `close`. - They return final path, remain active, and accept further commands. -- Cold replay completes the guard, returns the live session id, and accepts a later command in that session. -- Every action in ADR 0012's target-binding set has canonical identity evidence. - No unresolved `@ref` reaches disk. - Destination guards require verified landmark identity (#1349). - A rearranged-screen regression proves closed replay failure. -- Existing-target refusal preserves original bytes. + `open --save-script` on an existing session is rejected before app dispatch. +- An armed session publishes `open` plus target-annotated actions without `close`. + It returns the final path, remains active, and can continue to accept commands. +- The artifact replays from a cold start, completes its destination guard, returns the live session id, and accepts a subsequent command on that session. +- Every action in ADR 0012's existing target-binding command set has canonical identity evidence, and no unresolved `@ref` reaches disk. + Since #1349, the destination guard additionally requires verified recorded landmark identity. + A reshuffled-screen false-pass regression proves that replay fails closed. +- Existing-target refusal preserves the original bytes. `--force` replaces atomically. - Failed publication remains retryable. -- After PUBLISHED, ordinary actions remain usable and repeated `session save-script` fails. + A failed publish remains retryable. +- After PUBLISHED, later ordinary actions remain usable, and a repeated `session save-script` fails. Plain `open --relaunch` cannot re-arm. - `open --save-script` rejects before app dispatch. -- In ABORTED/PUBLISHED, `close --save-script[=]` rejects before platform close. - Plain `close` tears down without a write. - Unpublished ARMED close preserves existing publication behavior. -- On never-armed sessions, `close --save-script` rejects before platform close or filesystem work and names `open --save-script` as recovery. - Plain `close` still tears down without a write. + `open --save-script` is rejected before app dispatch. +- In ABORTED/PUBLISHED, `close --save-script[=]` is rejected before platform close. + Plain `close` tears down without writing. + The close of an unpublished ARMED recording preserves the current close-time publication behavior. +- On a never-armed session (2026-08-02 amendment), `close --save-script` is likewise rejected before platform close or filesystem work. + The refusal names `open --save-script` as the recovery. + Plain `close` still tears down without writing. The rejected request does not delete the session. -- Descriptor completeness tests classify every recordable request's mutation effect, including request-sensitive subcommands. - Only that trait controls destination-guard order. -- Repair-armed sessions refuse this action without repair-state changes. -- CLI help documents `--record-as` and literal unparameterized fill/type content. +- Descriptor completeness tests classify the mutation effect of every recordable request, including request-sensitive read-only/mutating subcommands. + Destination-guard ordering consumes only that trait. +- Repair-armed sessions refuse this action without a change to repair state. +- CLI help documents ADR 0017's `--record-as` workflow. + CLI help warns that unparameterized fill/type inputs remain literal script content. - Provider-backed integration scenarios cover the public daemon route. - Live iOS and Android runs prove artifact contents and post-save session behavior on real backends. + Live iOS and Android runs prove the saved artifact and the post-save session behavior on real backends. From 8167e2fc51058778d3bf952ea7c825a341eb11f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 46/55] docs(adr): restore the frozen meaning in ADR 0017 Restore the absolute "no secret-to-name table" rule and fix the "until its resolution" referent (#1348). Rewrite the glance bullets as statements that match the body's conditions, including the full session-scoped echo guarantee. Turn the PR's imperatives in landmark and action modes back into descriptive statements, and restore dropped "only", "re-serialize", "the literal is redacted either way", and "rather than an authoring convention". --- .../adr/0017-parameterized-recorded-inputs.md | 292 ++++++++++-------- 1 file changed, 158 insertions(+), 134 deletions(-) diff --git a/docs/adr/0017-parameterized-recorded-inputs.md b/docs/adr/0017-parameterized-recorded-inputs.md index c6fae53de1..1dc84f51df 100644 --- a/docs/adr/0017-parameterized-recorded-inputs.md +++ b/docs/adr/0017-parameterized-recorded-inputs.md @@ -6,32 +6,42 @@ Accepted ## Rules at a glance -- Use fill-only `recordAs`, CLI `--record-as `, for explicit parameterized inputs. -- Send the literal to the app. Retain only `${VAR}` in the fill's response, recording, event, and script. -- Use uppercase `[A-Z_][A-Z0-9_]*` variable names. Reserve `AD_*` for runtime built-ins. -- Reject `--record-as` without armed recording or with `--no-record`. -- Never infer sensitivity or variable names from values, selectors, labels, or field names. -- Replay derives `recordAs` only for complete placeholder text tokens, preserving placeholders during recorded replay or repair. -- Protect later derived results and identity evidence from registered literal echoes under the session-scoped amendment's limits. -- Keep unparameterized fill/type inputs literal. Do not scan explicitly authored selectors or positional text. +- The fill-only `recordAs` input parameterizes a recorded fill value. + The CLI spells it `--record-as `. +- The live fill dispatch receives the literal. + The public response, recording state, session event, and published script of the fill use `${VAR}`. +- Variable names use replay's uppercase `[A-Z_][A-Z0-9_]*` grammar. + `AD_*` remains reserved for runtime built-ins. +- `--record-as` is rejected unless script recording is armed. + `--record-as` cannot be combined with `--no-record`. +- Sensitivity is never inferred from a selector, label, field name, or value shape. + The caller names each mapping explicitly. +- If replay dispatches an authored fill whose complete text token is a `${VAR}` placeholder, replay derives `recordAs=VAR`. + Thus, a replay that is recorded or repaired retains the original placeholder. + Embedded interpolation remains ordinary script input. +- For the lifetime of one recording session, no later recorded action's own result payload or `target-v1`/`targets-v1` identity evidence may re-serialize an app-rendered echo of an already-parameterized literal (#1398). + The explicit scope limits of the #1398 amendment apply. +- Unparameterized fill/type values remain literal script content. + An author's own explicitly typed selector or positional text is not scanned. ## Context Ordinary `.ad` recording persists executable inputs. -A literal `fill` value therefore enters the script, including authoring passwords or tokens. -Diagnostic/event redaction cannot protect an executable artifact because replay still needs its value. +A literal `fill` value therefore becomes part of the published script. +This includes any password or token supplied during authoring. +Diagnostic and event redaction cannot make that executable artifact safe, because replay still needs the value. Native replay already resolves `${VAR}` from `--env`/`-e` and `AD_VAR_*` immediately before dispatch. The missing boundary is authoring. -The app needs the literal, but recording state and every publication write must receive only the placeholder. +The live app needs the literal, while recording state and every publication write must receive only the placeholder. Issue [#1348](https://github.com/callstack/agent-device/issues/1348) defines this requirement. -ADR 0016 deliberately excluded secret-bearing journeys until its resolution. +ADR 0016's active-session publication deliberately excluded secret-bearing journeys until #1348 was solved. ## Decision -Add fill-only `recordAs` input. -CLI spells it `--record-as `. -Node and MCP fill contracts expose `recordAs`: +Add a fill-only `recordAs` input. +The CLI spells it `--record-as `. +The Node and MCP fill contracts expose it as `recordAs`: ```sh export AD_VAR_PASSWORD='' @@ -41,153 +51,167 @@ agent-device wait 'role="heading" label="Home"' agent-device session save-script ``` -Live fill dispatch receives the literal. -Its public response, recording state, session event, and script use `${PASSWORD}`. -Before replay dispatch, `AD_VAR_PASSWORD` or `--env PASSWORD=` supplies the value. +The live fill dispatch receives the literal. +Its public response, recording state, session event, and published script use `${PASSWORD}`. +Later, replay resolves the placeholder from `AD_VAR_PASSWORD` or `--env PASSWORD=` before the fill is dispatched. Variable names use replay's uppercase `[A-Z_][A-Z0-9_]*` grammar. `AD_*` remains reserved for runtime built-ins. -Without armed recording, `--record-as` is rejected. -It cannot combine with `--no-record`. -Ordinary fills without it retain literal recording behavior. -Sensitivity is never inferred from selector, label, field name, or value shape. +`--record-as` is rejected unless script recording is armed. +`--record-as` cannot be combined with `--no-record`. +Ordinary fills without the option keep literal recording behavior. +Sensitivity is never inferred from a selector, label, field name, or value shape. ### Mapping contract -The caller explicitly names the mapping. -Reused `PASSWORD` deterministically emits `${PASSWORD}`. -`API_TOKEN` emits `${API_TOKEN}`. -There is no secret-to-name table, literal comparison, or de-duplication state for naming. -This avoids a second retained secret copy for that purpose. -It also makes distinct-value names visible authoring decisions. +The caller names the mapping explicitly. +Reuse of `PASSWORD` deterministically emits `${PASSWORD}`. +Use of `API_TOKEN` emits `${API_TOKEN}`. +There is no secret-to-name table and no literal comparison or deduplication state. +This rule avoids the retention of a second copy of the secret. +It also makes distinct-value naming an authoring decision that is visible in the script. ### Data-flow boundary -The literal exists only on the live request path long enough to execute the interaction: +The literal exists only on the live request path, and only long enough to execute the interaction: 1. Before platform work, the request scope registers the literal as an explicit diagnostics secret. - This supplements key-name redaction because opaque values may not resemble secrets. -2. Platform interaction receives the literal. -3. Before response/event/recording retention, semantic result `text` becomes `${VAR}`. - Every occurrence in post-fill `refLabel` is parameterized. - Stable selector/ref provenance remains byte-for-byte unchanged. - Untrusted backend extras and nested settle output parameterize literal occurrences in keys and values. - Post-fill selector-chain candidates are dropped only if parsed `text`, `label`, or `value` terms semantically contain the literal. - Known response/settle schema fields define the structural boundary and retain names and provenance values. - For whitespace-only literals, an affected untrusted string or key collapses to the placeholder. - Inline replacement cannot distinguish sensitive whitespace from ordinary separators. -4. At `recordActionEntry`, the recorder reconstructs the fill's literal positional and writes only the placeholder into `SessionAction`. - It parameterizes semantic result and arbitrary backend/settle echoes again. - ADR 0012 `target-v1` evidence retains its structure. - Exact value-bearing accessibility labels become placeholders. - Identity fragments that merely contain the same characters remain unchanged. - The `recordAs` control flag is not serialized. + This is in addition to key-name redaction, because an opaque value need not look secret. +2. The platform interaction receives the literal. +3. Before response/event/recording retention, the semantic `text` field of the fill result becomes `${VAR}`. + Every occurrence in a post-fill `refLabel` is parameterized. + Stable selector/ref provenance is preserved byte-for-byte. + Untrusted backend extras and nested settle output parameterize every occurrence of the supplied literal in both keys and values. + Post-fill selector-chain candidates are dropped only when a parsed `text`, `label`, or `value` term semantically contains the literal. + Known response and settle schema fields are the explicit structural boundary, and they retain their names and provenance values. + Inline whitespace replacement cannot distinguish a sensitive value from ordinary separators. + Thus, an untrusted string or key that contains a whitespace-only literal collapses to the placeholder instead. +4. At `recordActionEntry`, the recorder reconstructs the literal positional of the fill and writes only the placeholder into `SessionAction`. + The recorder parameterizes the semantic result field plus arbitrary backend/settle echoes again. + ADR 0012 `target-v1` evidence keeps its structure. + Exact value-bearing accessibility labels become the placeholder. + Identity fragments that merely contain the same characters are not rewritten. + The `recordAs` control flag itself is not serialized into the script. Parameterization is idempotent. - The second pass preserves existing placeholders even if the literal is `$` or part of the variable name. -5. The ADR 0012/0016 writer receives an already-parameterized action. - Selector provenance, portability checks, same-directory temp write, and atomic publication remain unchanged. + This second pass preserves placeholders inserted at the response boundary, even when the literal is part of the variable name of the placeholder or is `$`. +5. The existing ADR 0012/0016 writer receives an already-parameterized action. + Its selector provenance, portability checks, same-directory temp write, and atomic publication algorithm are unchanged. -Target and temp files use the same prepared script string. -After this pre-publication boundary, neither can contain the literal. +The target and the temp file are written from the same prepared script string. +Thus, neither can contain the literal once this pre-publication boundary has run. ### Replay and repair -Missing variables retain replay's explicit failure behavior. +Missing variables retain replay's existing fail-loud behavior. `${VAR}` resolution throws before the corresponding action is invoked. -For authored fills whose complete text token is `${VAR}`, daemon-only provenance derives `recordAs=VAR` for the live request. -It uses unresolved source action, independently of empty or whitespace resolved values. -Thus, recorded replay or repair retains the original placeholder instead of the expanded value. +If replay dispatches an authored fill whose complete text token is a `${VAR}` placeholder, replay's daemon-only provenance channel derives `recordAs=VAR` for that live request. +This derivation uses the unresolved source action, independently of whether the resolved value is empty or whitespace. +Thus, if the replay is itself being recorded or repaired, the recorder retains the original placeholder instead of serializing the expanded value. Embedded interpolation remains ordinary script input. Safe authoring emits one complete placeholder token. ### Session-scoped echo protection (amendment, #1398) -> **Status: accepted.** This amendment extends fill-step protection to recording-session scope under [#1398](https://github.com/callstack/agent-device/issues/1398). - -The original boundary protects only the originating fill request and recorded entry. -After [#1349](https://github.com/callstack/agent-device/issues/1349), later read-only actions independently record `target-v1` evidence and results from app-rendered state. -These include `wait` landmark evidence, `is`, and `get`. -Without memory of the earlier literal, an unrelated unparameterized action could retain an app-rendered echo through: - -- The filled field's displayed value. -- A search result. -- A validation message. -- A confirmation label. -- A destination landmark label. - -**The guarantee now applies to the recording session.** -During that session, no later action's result or `target-v1`/`targets-v1` evidence may serialize an app-rendered echo of an already-parameterized literal. -The explicit limits below apply. - -`SessionState.recordedFillLiterals` provides the minimum explicit ephemeral state needed to recognize later echoes. -This in-memory `Map` receives the same pair already computed by `fill --record-as` for its own boundary. -It adds one entry per parameterized fill, only after that fill's entry is recorded. -It never serializes to scripts, session event logs, or diagnostics. -Only its owning recorder can read it. -It disappears with the session. - -This does not reverse the Mapping contract's prohibition on a secret-to-name table. -That prohibition concerns state used to infer or de-duplicate variable names by comparison with earlier values. -`recordedFillLiterals` never selects names. -Every placeholder is exactly the author's chosen name. -The map only recognizes that named literal in later unrelated evidence. -It provides redaction, without a naming or comparison API. - -**Data classes receive two treatments:** - -- **Result/event/backend-output fields.** These display values are never compared during replay. - Content-aware substring redaction reuses the fill boundary's recursive backend-output scrub for every literal registered so far. - Each string receives one placeholder-safe left-to-right pass, not sequential full-string passes per literal. - Sequential passes could match a later literal inside an earlier inserted placeholder. +> **Status: accepted.** This amendment changes the data-flow boundary above from fill-step-scoped to recording-session-scoped, per issue [#1398](https://github.com/callstack/agent-device/issues/1398). + +The boundary above protects only the request/recording path of the originating fill itself. +After [#1349](https://github.com/callstack/agent-device/issues/1349), a later read-only action records `target-v1` identity evidence and a result payload independently. +These read-only actions are the landmark-mode evidence of `wait`, `is`, and `get`. +That capture comes from whatever the app renders at that later step. +The capture has no memory of the literal of an earlier fill. +Thus, an app-rendered echo of the literal can re-enter session state and publication through a completely different, unparameterized action. +This can occur even though the originating fill was parameterized. +Examples of such an echo: + +- The displayed value of the filled field itself. +- A search result, validation message, or confirmation label that contains the literal. +- A destination landmark whose accessible label includes the literal. + +**The guarantee is now recording-session-scoped, not fill-step-scoped.** +For the lifetime of one recording session, no later recorded action's own result payload or `target-v1`/`targets-v1` identity evidence may re-serialize an app-rendered echo of a literal that the session already parameterized. + +The mechanism is the smallest explicit, ephemeral state that can recognize a later echo. +This state is `SessionState.recordedFillLiterals`, an in-memory `Map`. +Only the SAME (literal, placeholder) pair that a `fill --record-as` entry already computes for its own boundary above populates the map. +The map gets one entry for each parameterized fill, added only after the entry of that fill itself has been recorded. +The map is never serialized: not to the script, the session event log, or diagnostics. +The map has no read API beyond the recorder that owns it. +The map disappears with the session. + +This is not a reversal of the "no secret-to-name table" rule in the Mapping contract above. +That rule rejects the retention of state to influence a *naming* decision. +An example is a comparison of a new `--record-as` value with prior values to infer or deduplicate a variable name. +`recordedFillLiterals` never informs a naming decision. +The placeholder of every entry is exactly the name that the author already chose. +The map is consulted only to recognize that same, already-named value when it reappears in unrelated later evidence. +It is a redaction lookup, not a secret registry with a naming or comparison API. + +**Two treatments, by data class:** + +- **Result/event/backend-output fields** (display data, never compared at replay). + These fields get content-aware substring redaction. + This redaction reuses the same recursive backend-output scrub that the fill boundary already applies to its own entry. + The redaction is generalized over every literal registered so far in the session. + The substitution itself is one placeholder-safe left-to-right pass over each string, not N sequential full-string passes. + A naive sequential pass over the same value can corrupt the just-inserted placeholder of an EARLIER pair. + This occurs when the literal of a LATER pair happens to be a substring of that placeholder. For example, register `somethinglong -> ${ABC}`, then `ABC -> ${OTHER}`. - Reprocessing `${ABC}` could produce `${${OTHER}}`. - Instead, the single pass tries registered literals longest-first at each position. - A shorter literal cannot partially consume a longer one, such as a username that prefixes a password. - The pass never scans emitted text again. - Thus, one pair's literal cannot match inside another pair's inserted placeholder in either direction. -- **`target-v1`/`targets-v1` identity evidence.** Replay compares `label`, `ancestry[].label`, and `scrollRegion.label` against the live tree. - These fields must never receive silent text substitution while retaining a trustworthy identity claim. - The live app legitimately renders the real value again. - A placeholder in trusted recorded identity therefore could not verify correctly. - Apply these rules instead: - - **Landmark mode (`wait`).** Treat an echo like #1349's identity-empty case. - Record no annotation and retain selector-existence semantics. - ADR 0016 requires `verification: "verified"` for destination guards. - A landmark whose only identity echoes a parameterized value therefore stops qualifying. - `session save-script` refuses it with existing "record a selector-targeted wait on a labeled or id-bearing landmark" recovery. - This enforces the motivating scenario's correction to stable `Apps` landmark selection. - - **Action mode (`get`, `is`, mutating element-targeting actions).** ADR 0012/0016 require identity evidence and forbid silent omission. - Never drop echo-bearing identity here. - Redact literal-bearing labels to placeholders through content-aware substring replacement. - Unlike the originating fill's exact-match rule, cross-step echoes commonly include surrounding app text, such as "Welcome, ``". - Downgrade `verification` to `"unverifiable"`, as decision 3's writer-parser invariant does for oversized payloads. - Replay then fails explicitly with `identity-unverifiable`. - It neither weakens to selector-only matches nor claims an identity label that cannot match again. + A second pass over a value already rewritten to `${ABC}` matches "ABC" *inside* that token, and produces `${${OTHER}}`. + The single pass tries every registered literal longest-first at each position. + Thus, one registered value that is a substring of another (a username that is a prefix of a password) is never partially consumed by the shorter pair. + The single pass also never re-scans text that it has already emitted. + Thus, no literal can ever be matched inside the placeholder token of another pair, in either direction. +- **`target-v1`/`targets-v1` identity evidence** (`label`, `ancestry[].label`, `scrollRegion.label`: the fields that replay's own classification compares). + This evidence is never silently text-substituted while it still claims a trustworthy identity. + Replay compares recorded identity against the *live* tree. + The live tree legitimately re-renders the real value again at replay time. + Thus, a placeholder written into a recorded identity field could never verify correctly. + Instead: + - **Landmark mode (`wait`).** An echo is treated exactly like the existing identity-empty case of #1349. + No annotation is recorded, and the wait keeps its selector-existence semantics. + ADR 0016's destination guard requires `verification: "verified"`. + Thus, a landmark whose only identity is an echo of a parameterized value simply stops qualifying as a guard. + `session save-script` refuses it with the existing "record a selector-targeted wait on a labeled or id-bearing landmark" recovery. + This reproduces the real resolution of the motivating scenario (a change of the guard to the stable `Apps` landmark). + It makes that resolution an enforced outcome, not an authoring convention. + - **Action mode (`get`, `is`, mutating element-targeting actions).** ADR 0012/0016 require identity evidence for every element-targeting recorded action and forbid silently dropping it. + Thus, an echo here is never dropped. + The literal-bearing label(s) are redacted to the placeholder. + This redaction is content-aware and substring-based, unlike the exact-match fill-boundary redaction. + The reason is that app-authored text typically surrounds a cross-step echo, such as "Welcome, ``". + `verification` is also downgraded to `"unverifiable"`. + This is the same fail-closed downgrade that the writer-parser invariant of decision 3 already uses for an oversized payload. + That downgrade makes the replay of the action fail loudly (`identity-unverifiable`). + Replay does not silently weaken the action to selector-only matching, and the script does not publish a label that could never match again. **Explicit scope limits:** -- Empty or whitespace-only resolved values do not enter the session-wide registry. - They retain existing fill-step protection only. - They disclose no distinctive content through echoes. - Collapsing arbitrary later strings would impose disproportionate readability costs and corrupt incidental substrings. -- The originating fill's entry uses only its existing exact-match fill-boundary pass for the new pair. - It never uses that pair in the coarser session-wide substring pass. - The session-wide guarantee covers later distinct actions, as #1398 specifies. -- Explicitly authored selectors and positional text are not scanned. - Only derived resolved evidence and results receive protection. - A secret typed into a new selector remains literal script content, like unparameterized fill/type input. -- The registry key is the literal. - If distinct `--record-as` names share a typed value, later echoes cannot identify their source fill. +- A whitespace-only or empty resolved fill value is excluded from the session-wide registry. + It keeps only the existing fill-step-scoped protection above. + The collapse of arbitrary later strings on a value with no discriminating content has a disproportionate readability cost. + Such a value reveals nothing distinctive if echoed. + The collapse would also corrupt unrelated short incidental substrings throughout the rest of the recording. +- The originating fill's own recorded entry is protected only by the existing exact-match fill-boundary pass. + That entry never goes through the coarser, substring-based session-wide pass with the pair that it just contributed. + The session-wide guarantee is for later, distinct actions, exactly as #1398 frames it. +- An author's own explicitly typed selector or positional text is not scanned. + Only derived, resolved evidence and result payloads are scanned. + An author who types the secret directly into a new selector makes the same choice as an unparameterized fill/type value. + That text remains literal script content. +- The registry is keyed by literal. + Two distinct `--record-as` names can share the same typed value (for example, a password/confirm-password pair). + From the perspective of a later echo, these names are genuinely indistinguishable, because the literal alone cannot tell which fill produced it. The first-registered name wins deterministically. - This changes placeholder attribution only, never literal protection. + The literal is redacted either way. + Thus, this affects only the placeholder name to which a later echo is attributed, never whether the value is protected. ## Consequences -- Secret-bearing login/bootstrap scripts can safely use ADR 0016 active publication under these rules. -- Callers must opt in for every sensitive fill. - Unparameterized fill/type values remain literal content, with a help warning. -- This decision adds no parameterization for `type` or mutating `find ... fill|type`. - If needed, they require separate surface and provenance design. +- Secret-bearing login/bootstrap scripts can use ADR 0016 active publication safely. +- The caller must opt in for every sensitive fill. + Unparameterized fill/type values remain literal script content, and help warns accordingly. +- This decision does not add `type` and mutating `find ... fill|type` parameterization. + If needed, they require their own surface and provenance design. - No new publication format, variable store, or sensitivity heuristic is introduced. - The #1398 amendment adds one ephemeral non-serialized per-session redaction map. - It has no naming or lookup-by-name API and never becomes a persistent secret registry. + The #1398 amendment adds one ephemeral, non-serialized, per-session redaction map with no naming or lookup-by-name API. + This map is never a persistent secret registry. From c947b7521427d5f29dd75f5590d3d3642b6780a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:50:56 +0200 Subject: [PATCH 47/55] docs(adr): restore the frozen meaning in the ADR index Restore "replay healing" and the ADR 0032 "revisit triggers" term in the index. Restore dropped nouns that carried scope ("error handling", "semantics", "cleanup fix", "where ... enforced", "that the table refused"). Scope the waiver/evidence-gap convention to the Status section again. --- docs/adr/README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/adr/README.md b/docs/adr/README.md index c4dd48c3de..f0384d98e0 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -2,7 +2,7 @@ | ADR | Read when you touch… | | --- | --- | -| [0001 Provider-First Integration Scenarios](0001-provider-first-integration-scenarios.md) | integration test strategy, provider transcripts, the scenario harness | +| [0001 Provider-First Integration Scenarios](0001-provider-first-integration-scenarios.md) | integration testing strategy, provider transcripts, the scenario harness | | [0002 Persistent Platform Helper Sessions](0002-persistent-platform-helper-sessions.md) | helper process lifecycle, keep-alive semantics | | [0003 Daemon Command Registry](0003-daemon-command-registry.md) | daemon routing, request-policy traits | | [0004 iOS Snapshot Backend Strategy](0004-ios-snapshot-backend-strategy.md) | snapshot capture plans, backend fallbacks, quality verdicts, the coordinate space a capture publishes | @@ -13,10 +13,10 @@ | [0009 Apple Platform Consolidation](0009-apple-platform-consolidation.md) | Apple platform family, apple/appleOs axes, the apple-leak guard | | [0010 Error system conventions](0010-error-system.md) | error codes, hints, normalizeError, typed error signals | | [0011 Interaction Guarantee Contract](0011-interaction-guarantee-contract.md) | interaction dispatch paths, fast paths, guards, the guarantee matrix, parity tables | -| [0012 Interactive Replay](0012-interactive-replay.md) | replay repair/`--update`, diagnostic resolution disclosure, bounded `.ad` target-binding evidence, bounded divergence wire/errors, plan-bound replay-only `--from`, and agent-supervised re-record repair ("heal-by-doing") | -| [0013 Unified Gesture Plans](0013-unified-gesture-plans.md) | gesture API and routes, contact topology, multi-touch geometry, native pointer injection, two-finger pan | +| [0012 Interactive Replay](0012-interactive-replay.md) | replay healing/`--update`, diagnostic resolution disclosure, bounded `.ad` target-binding evidence, bounded divergence wire/error handling, plan-bound replay-only `--from` semantics, and agent-supervised re-record repair ("heal-by-doing") | +| [0013 Unified Gesture Plans](0013-unified-gesture-plans.md) | gesture API/routing, contact topology, multi-touch geometry, native pointer injection, two-finger pan | | [0014 Session Ref-Frame Lifetime](0014-session-ref-frame-lifetime.md) | ref authorization epochs, complete/partial issuance, pre-side-effect expiration, replay/batch compatibility, and cross-platform stale-mutation policy | -| [0015 Direct Maestro Compatibility Engine](0015-direct-maestro-engine.md) | Maestro YAML parse and execution, compatibility observation policy, conformance, performance gates, gesture integration | +| [0015 Direct Maestro Compatibility Engine](0015-direct-maestro-engine.md) | Maestro YAML parsing/execution, compatibility observation policy, conformance, performance gates, gesture integration | | [0016 Active-Session Script Publication](0016-active-session-script-publication.md) | publication of an armed open-to-destination `.ad` script that retains its live session | | [0017 Parameterized Recorded Inputs](0017-parameterized-recorded-inputs.md) | safe authoring of sensitive fill inputs as `${VAR}` placeholders across recording, replay, and repair | | [0018 Unified Request Event Journal (Proposed)](0018-unified-event-journal.md) | event/diagnostic vocabulary, journal scopes and sinks, progress-channel separation, observability-only state | @@ -25,16 +25,16 @@ | [0021 Simlock-Leased Devices Through Ordinary Local Runtimes](0021-host-simlock-managed-device-allocation.md) | operation of a Simlock-leased simulator or emulator, the reason for no agent-device allocator integration, and the withdrawn in-process Host design | | [0022 Daemon — Platform Runtime Coupling Audit and Ownership Ratchets](0022-daemon-platform-runtime-coupling.md) | daemon imports of root `platform-runtime-*` modules, the coupling ban that replaces R76 edge classification, the R75 session-authority ratchet, entry-to-platform hop routes and roles | | [0023 End-State Entry-to-Platform Hop Trace](0023-end-state-hop-trace.md) | the definition of hop counts, ordered `press`/Android and dual-arm `snapshot`/iOS chains, per-hop roles, and the deletion test for numbers in ADR 0019 and ADR 0022 | -| [0024 Screen Recording — One Coordinator, Two Facts (Proposed)](0024-screen-recording-provable-signal.md) | `record start`/`stop` lifecycle, separate export-exists / recorder-stopped facts, backend stop-and-collect contract, coordinator recovery, the ADR 0019 §5 amendment, and cleanup after failed finish | +| [0024 Screen Recording — One Coordinator, Two Facts (Proposed)](0024-screen-recording-provable-signal.md) | `record start`/`stop` lifecycle, separate export-exists / recorder-stopped facts, backend stop-and-collect contract, coordinator recovery, the ADR 0019 §5 amendment, and the failed-finish cleanup fix | | [0025 Foldable Apple Panels — Capture the Lit Panel](0025-foldable-apple-panels.md) | iPhone Duo outer/inner panels, CoreDevice display info as sole screen authority, black-panel capture from `simctl`'s implicit display default, derived (never asserted) pose, `fold` pose control, and the Android emulator posture amendment | | [0026 Scroll Clip Authority — Ownership Is the Parent Edge](0026-scroll-clip-authority.md) | iOS scroll indicators and the visible band, parent-edge ownership instead of ancestor traversal, permitted node removal from interactive snapshots, and ejection dispositions | | [0027 Descriptor Root Size vs Eager-Closure Budget](0027-descriptor-root-vs-eager-closure-budget.md) | accepted: the eager-closure budget permits only a statement-neutral hub split (containment + added-module novelty + flat top-level-statement weight); `packages/command-registry/src/registry.ts` may be split | | [0028 Capability-Family Cell Vocabulary — One Runtime Source (Proposed)](0028-capability-family-cell-vocabulary.md) | capability operation family additions, `UnavailablePlatformRuntimeFacts` / `UNAVAILABLE_CELLS`, `INTERACTOR_OPERATIONS`, and the reason for repeated changes across eight packages | -| [0029 Daemon Policy](0029-daemon-policy.md) | `AGENT_DEVICE_DAEMON_POLICY`, limits on daemon commands, devices, or device shutdown, and operator-rule enforcement for batch/replay steps | +| [0029 Daemon Policy](0029-daemon-policy.md) | `AGENT_DEVICE_DAEMON_POLICY`, limits on daemon commands, devices, or device shutdown, and where operator rules are enforced for batch/replay steps | | [0030 Process Lock Exclusion](0030-process-lock-exclusion.md) | process-lock publication/reclaim/release, retained mutation guards, and the single-protocol upgrade boundary | | [0031 macOS Native App Backend](0031-macos-native-app-backend.md) | `AGENT_DEVICE_MACOS_APP_BACKEND`, macOS app sessions without XCTest Automation Mode, the accessibility-only pointer action rule, and the ghost cursor | -| [0032 Layering Graph Engine](0032-layering-graph-engine.md) | replacement evaluation for `scripts/layering/` graph rules with dependency-cruiser or fallow `boundaries`, the reason for a shared replaceable import model, measured edge-set and parity gaps, and reevaluation conditions | -| [0033 Collocation decision table](0033-collocation-decision-table.md) | moves or additions for files in umbrella #3276 §5, renewed proposals for refused moves, and a zone's collocation decisions | +| [0032 Layering Graph Engine](0032-layering-graph-engine.md) | replacement evaluation for `scripts/layering/` graph rules with dependency-cruiser or fallow `boundaries`, the reason that graph-based layering checks share one replaceable import model, the measured edge-set and parity gaps, and the revisit triggers | +| [0033 Collocation decision table](0033-collocation-decision-table.md) | moves or additions for files in umbrella #3276 §5, renewed proposals for a move that the table refused, and a zone's collocation decisions | | [0034 Provider plugin registry](0034-provider-plugin-registry.md) | provider plugins, the `agent-device/plugins` contract, plugin sources and order, duplicate provider ids, plugin failure isolation, the daemon plugin inventory and `daemon status` | ADRs record rationale. @@ -51,5 +51,5 @@ Use these shape conventions to keep ADRs inexpensive to consult: Put them below the summary. - **Process history is deleted once complete.** Keep completed migration plans, per-step implementation tables, and point-in-time status logs only in git history. In Status, keep one line that states this deletion. - Retain accepted waivers and evidence gaps that still apply. + In Status, also keep any accepted waiver or evidence gap that is still relevant. - If a gate enforces a rule, retain its rationale and a gate pointer instead of repeated rule details. From 7246192bc3a77b850528fabb9c01bfe48f34d610 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Fri, 16 Oct 2026 17:56:15 +0200 Subject: [PATCH 48/55] docs(adr): scope landmark mode to wait in ADR 0017 The STE rewrite read as if `is` and `get` also record landmark-mode evidence. The base lists three read-only actions: `wait` in landmark mode, `is`, and `get`. Found in review round 2. --- docs/adr/0017-parameterized-recorded-inputs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0017-parameterized-recorded-inputs.md b/docs/adr/0017-parameterized-recorded-inputs.md index 1dc84f51df..a5ee3e419d 100644 --- a/docs/adr/0017-parameterized-recorded-inputs.md +++ b/docs/adr/0017-parameterized-recorded-inputs.md @@ -116,7 +116,7 @@ Safe authoring emits one complete placeholder token. The boundary above protects only the request/recording path of the originating fill itself. After [#1349](https://github.com/callstack/agent-device/issues/1349), a later read-only action records `target-v1` identity evidence and a result payload independently. -These read-only actions are the landmark-mode evidence of `wait`, `is`, and `get`. +These read-only actions are `wait` with landmark-mode evidence, `is`, and `get`. That capture comes from whatever the app renders at that later step. The capture has no memory of the literal of an earlier fill. Thus, an app-rendered echo of the literal can re-enter session state and publication through a completely different, unparameterized action. From 965723be928f24bc0b7761cefc7d3a0ebc7c86b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Sat, 17 Oct 2026 08:02:55 +0200 Subject: [PATCH 49/55] docs(adr): scope the ADR 0002 fail-closed rule to Android snapshots The base rule applies to the snapshot helper and forbids the uiautomator dump substitute. The glance bullet dropped both. https://github.com/callstack/agent-device/pull/3729#discussion_r4296391183 --- docs/adr/0002-persistent-platform-helper-sessions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0002-persistent-platform-helper-sessions.md b/docs/adr/0002-persistent-platform-helper-sessions.md index 779a844512..276fa8f198 100644 --- a/docs/adr/0002-persistent-platform-helper-sessions.md +++ b/docs/adr/0002-persistent-platform-helper-sessions.md @@ -17,7 +17,7 @@ Accepted - On Android, snapshot capture, gesture viewport resolution, and planned-touch injection share one bundled automation helper. - On Android, after a failed session command, retry one-shot only for idempotent reads, and only after the failed session has stopped. - On Android, return a session gesture failure directly. Do not retry it one-shot. -- If the packaged Android helper is unavailable or invalid, agent-device must fail closed. +- For Android snapshots, if the packaged helper is unavailable or invalid, agent-device must fail closed. It must not substitute the legacy `adb uiautomator dump` snapshot engine. ## Context From 622e936eb4a019e7dfe44c0339aa3bc898492934 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Sat, 17 Oct 2026 08:02:55 +0200 Subject: [PATCH 50/55] docs(adr): state the ADR 0007 lease allow list as fail closed https://github.com/callstack/agent-device/pull/3729#discussion_r4296391079 --- docs/adr/0007-remote-device-leases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0007-remote-device-leases.md b/docs/adr/0007-remote-device-leases.md index 96b0abf5f5..b036a35427 100644 --- a/docs/adr/0007-remote-device-leases.md +++ b/docs/adr/0007-remote-device-leases.md @@ -268,7 +268,7 @@ Thus, `batch` steps and `replay` actions are confined when they enter admission The confinement has these parts: - An allow list of commands: the commands whose command registry descriptor declares `appLease: 'allowed'`. - Later commands are refused. + A command without that declaration is refused, including a command that is added later. Of the commands that lease admission otherwise exempts, only `lease_heartbeat` and `lease_release` declare it. - `open` and `close` of the leased bundle only. - The `app` surface only. From 632b210e6d68ff0d63f29f4a08b56dc0fbf8186a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Sat, 17 Oct 2026 08:02:55 +0200 Subject: [PATCH 51/55] docs(adr): name the retired table parity test in ADR 0008 The retired gate compared derived tables byte for byte. src/__tests__/command-descriptor-parity.test.ts compares name sets. https://github.com/callstack/agent-device/pull/3729#discussion_r4296391165 --- docs/adr/0008-command-descriptor-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0008-command-descriptor-registry.md b/docs/adr/0008-command-descriptor-registry.md index dd3b3165f0..167d9e5d58 100644 --- a/docs/adr/0008-command-descriptor-registry.md +++ b/docs/adr/0008-command-descriptor-registry.md @@ -12,7 +12,7 @@ Completed process history is in git history. > The descriptor's other derived projections remain accepted. > > **Parity gate retired (2026-09).** The completed migration deleted every hand-authored table, so a parity test has no second table to compare. -> No parity test exists. +> No byte-for-byte table parity test exists. > Do not add or search for one. > Compile-time guards (`CommandOwnerFileClaimsAreComplete`) and literal pin-set tests now enforce totality. > Those tests compare name sets instead of two tables. From aefc91507c4100de70f423d1c3c8029ec9f8095b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Sat, 17 Oct 2026 08:04:51 +0200 Subject: [PATCH 52/55] docs(adr): keep the ADR 0016 ABORTED arming rule in the present tense The base states the rule as current. The rewrite put it in the past tense and added a sentence that made it read as replaced. --- docs/adr/0016-active-session-script-publication.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/adr/0016-active-session-script-publication.md b/docs/adr/0016-active-session-script-publication.md index e45326f958..e0136ef5b4 100644 --- a/docs/adr/0016-active-session-script-publication.md +++ b/docs/adr/0016-active-session-script-publication.md @@ -126,11 +126,10 @@ For `close --save-script` in ABORTED, this ADR specifies a refusal that promises Thus, the gap also made an existing error message untrue. ABORTED is now terminal by construction. -This amendment routed `--save-script` arming through one rule that the publication projection owned. -That rule answered "not recording" for an ABORTED lifecycle on every surface that handled the flag. -These surfaces were the re-open builder, the close finalizer, and the recorded-action ingress. -Thus, the flag could no longer contradict the status. -The follow-up amendment below replaces the stored flag with a derivation. +`--save-script` arms recording through one rule that the publication projection owns. +That rule answers "not recording" for an ABORTED lifecycle on every surface that handles the flag. +These surfaces are the re-open builder, the close finalizer, and the recorded-action ingress. +Thus, the flag can no longer contradict the status. The aggregate likewise answers publication authorization. The writer asks one publication-blocked question. That question covers not-recording, an uncommittable or committed repair, and an ABORTED authoring lifecycle alike. From 55c3eb41492ad2ffbeae1c099eaf3169c60e8f3e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Sat, 17 Oct 2026 08:05:05 +0200 Subject: [PATCH 53/55] docs(adr): restore the ADR 0007 glance limits for cancellation and heartbeats --- docs/adr/0007-remote-device-leases.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/0007-remote-device-leases.md b/docs/adr/0007-remote-device-leases.md index b036a35427..b829062b1f 100644 --- a/docs/adr/0007-remote-device-leases.md +++ b/docs/adr/0007-remote-device-leases.md @@ -12,8 +12,8 @@ Accepted - Generated connection profiles must strip daemon and Metro bearer tokens. - A session cannot outlive its lease. Lease-end teardown clears the session's device claim even when the device no longer answers. - Report each lease that a request ended once, at response level. A client never sends a release after such a request. -- Admitted work that is still wanted preserves its lease. If the client hung up, the request neither defers expiry nor renews the lease. -- Before admission, protect long client phases with lease heartbeats. Renew under the window that the caller named, or else under the lease's existing window. +- Admitted work that is still wanted preserves its lease. If the client hung up, the request neither defers expiry past that cancellation nor renews the lease. +- Before admission, send lease heartbeats for as long as a client phase runs, for example an artifact upload. Renew under the window that the caller named, or else under the lease's existing window. - Before a pending human-control activation reports active, it fences new mutations and drains the mutations already admitted. - Only host administrators allocate `macos-app` leases. Tenant `lease_release` is refused. - Confine each request admitted under a `macos-app` lease to the leased app. From 4e88d451055c67eae217737ca948df9696aa239b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Sat, 17 Oct 2026 08:05:05 +0200 Subject: [PATCH 54/55] docs(adr): restore the ADR 0005 precondition referent --- docs/adr/0005-ios-runner-interaction-lifecycle.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0005-ios-runner-interaction-lifecycle.md b/docs/adr/0005-ios-runner-interaction-lifecycle.md index ba6c213e20..a165c91b6a 100644 --- a/docs/adr/0005-ios-runner-interaction-lifecycle.md +++ b/docs/adr/0005-ios-runner-interaction-lifecycle.md @@ -63,7 +63,7 @@ The record is written only after the `runnerFatal` check. Thus, sparse AX-fallback snapshots and `runnerFatal` payloads never refresh it. Snapshots and other read-only responses never count as a health signal. -This narrow skip is permitted because it meets the preflight-reduction precondition in the Consequences section: +This narrow skip is permitted now, because the preflight-reduction precondition in the Consequences section is met: - Coordinate-first activation removed the command-induced teardown trigger. - The lifecycle status journal plus the status-before-invalidate recovery is the teardown-surviving status surface. From 41506304f84b52a51bdf1da64a8f1a503d599438 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Sat, 17 Oct 2026 08:06:05 +0200 Subject: [PATCH 55/55] docs(adr): keep the ADR 0008 AgentDeviceClient import in the present tense src/commands/family/types.ts still imports AgentDeviceClient. That import is the reason for the retained Input/Result ownership rule. --- docs/adr/0008-command-descriptor-registry.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/adr/0008-command-descriptor-registry.md b/docs/adr/0008-command-descriptor-registry.md index 167d9e5d58..337fec37e5 100644 --- a/docs/adr/0008-command-descriptor-registry.md +++ b/docs/adr/0008-command-descriptor-registry.md @@ -104,7 +104,8 @@ The descriptor is the prerequisite for typed per-command results in ADR 0010, wh It also supplies the capability facet used by the platform-plugin work in ADR 0009. During the migration, the import-cycle inversion was the principal risk. -At that time, `command-catalog.ts` had approximately 95 importers, and the family facet imported `AgentDeviceClient`. +At that time, `command-catalog.ts` had approximately 95 importers. +The family facet imports `AgentDeviceClient`. Thus, the descriptor module must own the `Input`/`Result` types. The client must be derived as a view type, and a lint boundary enforces this.