Skip to content

fix(runner): refuse an iOS coordinate fill whose point names no text input - #3718

Merged
thymikee merged 3 commits into
mainfrom
maint/1721-coordinate-fill-fail-closed
Oct 16, 2026
Merged

thymikee merged 3 commits into
mainfrom
maint/1721-coordinate-fill-fail-closed

Conversation

@thymikee

@thymikee thymikee commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Summary

On the iOS degraded-XCTest-channel fill route — which ref, selector, and replay fills all reach as a point — the synthesized first-responder replacement has no element: with no text input under the point it still dispatched its focus tap and ran clear+type through the app-level first responder, so a fill aimed at empty space rewrote whichever field an earlier focus left active (#1721). The runner now refuses before the focus tap: NO_TEXT_INPUT_AT_POINT when the point identified no input, TEXT_INPUT_PROBE_UNAVAILABLE when the pre-tap lookup could not answer (a blind point is not an empty one). Each publishes its own error.details.reason with dispatched: no; nothing runs and no field changes. Eleven files, one command family, three commits (fix/test/fix); no declared guarantee in interaction-guarantees.ts changes.

agent-device fill 300 780 "text"   # point with no input → NO_TEXT_INPUT_AT_POINT, dispatched: no
agent-device fill 201 496 "text"   # point inside an input → unchanged behavior

Validation

Base 5007540a9, head 28c6cf1a6 (rebased; the red was captured pre-rebase at the same content). Same XCTest on an iPhone 17 Pro simulator:

  • base: testPenalizedCoordinateFillAtAPointWithNoTextInputRefusesBeforeItsTapAndKeepsTheFocusedField red — TEXT_INPUT_CLEAR_NOT_OBSERVED instead of the refusal, and the focused field's seeded "stale" was destroyed.
  • head: that test plus the failed-lookup (TEXT_INPUT_PROBE_UNAVAILABLE) and the 55-test fill/coordinate/synthesized-entry family green. The regression asserts the fixture's tap witness stays at presses=0; moving the guard after the focus tap turns it red with presses=1, so the pre-tap claim is pinned.

Live on a self-created simulator (production daemon + runner; the channel penalty forced by a local-only scratch build, reverted before commit; captured at 81601c1c8, identical route code at head): the empty-space fill refused with NO_TEXT_INPUT_AT_POINT / no_text_input_at_point / dispatched: no, the field held "stale", the witness held presses=0, and fill 201 496 on the real input still landed.

pnpm check:affected --run --head 28c6cf1a6: passed (exactCommit=true; format, lint, typecheck, layering, fallow, build, vitest-related, command-docs, docs-links, docs-prose, xctest-selection, packaged-runner-swift). CI obligations: ios.yml targeted runner lane selects the new tests at head. Resolved risks: the non-penalized route keeps its pre-existing untyped refusal (out of scope); the .xctestFallback unbound window predates this PR (see review reply); the live penalty was forced because a healthy simulator cannot degrade the XCTest channel on demand.

Fixes #1721

🤖 Generated with Claude Code

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread website/docs/docs/commands.md Outdated
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-16 17:02 UTC

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.30 MB 5.30 MB +4.5 kB
Package (unpacked) 5.30 MB 5.30 MB +4.5 kB
Package (download) 1.59 MB 1.60 MB +973 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 21.1 ms 20.8 ms -0.3 ms
CLI --help 58.4 ms 57.2 ms -1.2 ms

@thymikee

Copy link
Copy Markdown
Member Author

I reviewed 5b02140 and found one problem to fix before merge. The guard is good for fill <x> <y>, but the host fill backend sends x/y for every iOS fill, so fill @ref, fill <selector> and Maestro inputText now reach it too (interactions.ts, guard). If the centre of a ref or selector rect is not inside an XCTest text field, secure field, search field or text view, the fill now fails with NO_TEXT_INPUT_AT_POINT. Examples are an RN input exposed as Other, or a container ref whose tap focuses the inner field. Before this PR those fills worked through the first responder. The hint says "fill the field by ref or selector", which sends a caller who already used a ref back to the same refusal, and commands.md scopes the code to fill <x> <y> only. Could you make the refusal text and the docs line describe every route that reaches the guard, for example by telling the caller to aim inside the field's rect or run snapshot -i to pick the input node? Please also add a CHANGELOG line for ref, selector and replay fills under a degraded XCTest channel.

Not blocking: the new guard sits beside the predicate it qualifies, so in the .xctestFallback branch (line 79) shouldUseSynthesizedFirstResponderReplacement can still admit an unbound replacement. The window is narrow and predates this PR, and you can take or leave it. Requiring a bound identity in that predicate would cover it.

Could coordinateTapTextInputIdentityAt return the existing TextInputProbeOutcome (input, absent, unavailable) instead? Then .absent becomes NO_TEXT_INPUT_AT_POINT and .unavailable becomes its own typed probe-unavailable reason that does not dispatch. With the bound-identity check in shouldUseSynthesizedFirstResponderReplacement, the fallback route is covered without a second guard. Both owners already exist, so nothing has to change first, and the host table work can stay as is.

The cubic-dev-ai threads on the failed-lookup mapping and the missing tap-count assertion still apply, and so does the one lower-priority thread on the malformed comment. The first one is the same split as above.

CI is pending. Only the two Smoke Tests jobs have not passed, and they are queued, not failed. I do not expect them to hit the penalized XCTest channel, so I see nothing to attribute yet. There are no conflicts.

I did not run the XCTest red/green or the live simulator run, so those rest on the PR body. I also did not trace the daemon fill handler line by line to confirm that every ref and selector fill reaches the x/y backend. I inferred that from the interactor signature and interaction-guarantees.ts:407, and I could not judge how often the post-tap lookup fails in the fallback branch.

Before merge, please fix the hint and docs with the CHANGELOG line, and split probe-unavailable from no-input, then let the queued Smoke Tests finish.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread website/docs/docs/commands.md Outdated
@thymikee

Copy link
Copy Markdown
Member Author

Pushed 81601c1c8 addressing both blocking asks:

Split probe-unavailable from no-input. coordinateTapTextInputIdentityAt now returns a three-way CoordinateTextInputResolution (.input / .absent / .unavailable) instead of an optional, covering both blindness sources: a contained probe issue/exception and a candidate whose snapshot() fails transiently. .absent answers NO_TEXT_INPUT_AT_POINT; .unavailable answers the new TEXT_INPUT_PROBE_UNAVAILABLE, both before the focus tap (dispatched: no), each with its own details.reason and a hint that says the blind refusal does not mean the point is empty. The failed-lookup XCTest now pins TEXT_INPUT_PROBE_UNAVAILABLE, and the host table pins the pair as distinct (runner-error-classification.test.ts).

Routes and hint. Agreed on the reach: fill @ref, selector and Maestro inputText all arrive as x/y, so the guard covers them too. The NO_TEXT_INPUT_AT_POINT hint now says a ref/selector fill aims at the centre of the rect it resolved and that an outer container or non-input node can miss the field — pick the input node in snapshot -i and aim inside its rect. commands.md drops the fill <x> <y>-only qualifier and names all four routes plus the degraded-channel condition, and the CHANGELOG carries an Unreleased line covering ref, selector and replay fills.

Non-blocking predicate idea — took the probe split, left the predicate. shouldUseSynthesizedFirstResponderReplacement still admits the .xctestFallback unbound replacement. That route reaches the predicate only after the synthesized tap failed and focusTextInputForTextEntry re-resolved through its own lookup, so requiring a bound identity there changes a pre-existing route whose evidence (tap witness, stabilization) is not the point probe's. The #1721 damage needs "point named no input AND typed anyway", and after this push every path where the pre-tap probe can answer is guarded or bound. I'd rather cover the fallback in a follow-up with its own red test; the two-line table change without one would just move the guess. The probe split is the shape you sketched; the predicate becomes its natural second commit.

Live evidence at the new head: same forced-penalty scratch recipe — fill over empty space with a focused field refuses NO_TEXT_INPUT_AT_POINT with the field unchanged; the witness assertion (presses=0) proves the tap never dispatched; a bound fill still lands.

…input

On the penalized coordinate route the replacement has no element, so its
clear and its typed text reach the app-level first responder. Aimed at
empty space while an earlier field still holds focus, that rewrote the
earlier field. The runner now answers NO_TEXT_INPUT_AT_POINT before the
focus tap, and the host publishes details.reason no_text_input_at_point
with dispatched: no.

Fixes #1721
…s tap

The refusal's dispatched: no claim needs device-side evidence: the fill
aimed at a button that counts presses, so a refusal that tapped first
moves the counter even when the field comes through unchanged. Also say
that the runner identified no input at the point, since a lookup that
cannot answer reports the same code as an empty point.

Fixes #1721
The pre-tap lookup returned nil for both, so one refusal carried two
claims. coordinateTapTextInputIdentityAt now answers input, absent, or
unavailable; absent refuses with NO_TEXT_INPUT_AT_POINT and unavailable
refuses with TEXT_INPUT_PROBE_UNAVAILABLE, both before the focus tap.
The hint and docs name every route that reaches the guard, including
ref, selector and replay fills, whose aim point is the centre of the
rect they resolved.

Fixes #1721
@thymikee
thymikee force-pushed the maint/1721-coordinate-fill-fail-closed branch from 81601c1 to 28c6cf1 Compare October 16, 2026 15:31
@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the update. I reviewed 28c6cf1 and the code looks good. The two blocking asks from the earlier review (#3718 (comment)) are fixed. The blind lookup now has its own response, textInputProbeUnavailableResponse in TypeExecution.swift. The hint, docs and CHANGELOG now cover coordinate, ref, selector and replay fills under a degraded channel. All 20 checks pass at this commit, and I know of no conflicts. Nothing else blocks merge.

All four earlier cubic-dev-ai threads are fixed at this commit, so please resolve them: #3718 (comment) (probe-unavailable split), #3718 (comment) (tap witness test), #3718 (comment) (comment rewrite) and #3718 (comment) (docs and CHANGELOG route coverage).

I did not run the XCTest red/green or the live forced-penalty simulator run. Both rest on the PR body. The author captured the live run before the rebase and reports the route code is identical at this commit. I could not run the TEXT_INPUT_PROBE_UNAVAILABLE path live, so only the injected-issue XCTest covers it. The unbound-replacement window in the .xctestFallback path predates this PR. The follow-up with its own red test is fine.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 16, 2026
@thymikee
thymikee merged commit ffee792 into main Oct 16, 2026
20 checks passed
@thymikee
thymikee deleted the maint/1721-coordinate-fill-fail-closed branch October 16, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fail closed for unresolved synthesized coordinate fill on iOS

1 participant