Skip to content

fix(ios): alert dismiss picks a listed decline, never a guessed one - #3715

Merged
thymikee merged 8 commits into
mainfrom
fix/alert-decline-choice
Oct 16, 2026
Merged

thymikee merged 8 commits into
mainfrom
fix/alert-decline-choice

Conversation

@thymikee

@thymikee thymikee commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Summary

Closes #3627. On the Apple XCUITest runner, alert dismiss chooses a recognized decline action, including "Ask App Not to Track", or refuses with ALERT_BUTTON_NOT_FOUND and details.dispatched: no. Unknown and localized choices receive no guessed tap. The exact "dismiss popup" marker retains its close-element behavior; accept-side vocabulary is unchanged.

Validation

Published head: 9203a7a4066fec99f54ef32710350bfa6462f9d7, rebased onto f633525b1, mergeable.

  • The rebase preserves this issue's alert-choice source, native choice tests and shared alert contract byte-for-byte against 2ab0df6d0, so the live proof below transfers to this commit by construction. Conflicts retain both upstream text-input refusal codes and this branch's alert refusal code. A fresh isolated full-access Apex review in its own checkout at 9203a7a40: approve, no blocking findings; it independently verified union fidelity, the frozen decline-only/accept vocabularies, the exact-marker exception, and contract coherence after the upstream reason-table refactor.
  • Direct exact-head gate run-LlH55N is queued behind live repository validation; no passing receipt is claimed yet. The owner runs the 11 alert XCTests on-device after the broad gate; that receipt will be appended here.
  • Three new XCTests are named-mutation invariant pins of the choice rule, not revert-fail pins; the revert path is covered by the e2e canaries.
  • Historical full CLI/system ATT evidence remains attributed to 2ab0df6d0: one decline activation, TCC denied and the disposable app's denied callback. The affirmative-only fixture refused through CLI/daemon/runner with dispatched: no, preserved its hint and logged zero activations. No global privacy reset was used.

The readiness label remains present, but the exact-head gate, native tests and current-head CI are pending. The earlier head's green CI and gate do not certify this rebased commit.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 13 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/commands/capture/alert.ts Outdated
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-16 17:43 UTC

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.30 MB 5.31 MB +3.9 kB
Package (unpacked) 5.30 MB 5.31 MB +3.9 kB
Package (download) 1.60 MB 1.60 MB +1.1 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 22.9 ms 23.3 ms +0.4 ms
CLI --help 63.5 ms 62.7 ms -0.9 ms

@thymikee
thymikee force-pushed the fix/alert-decline-choice branch from 38d737f to 2ab0df6 Compare October 16, 2026 13:40
@thymikee

Copy link
Copy Markdown
Member Author

The code change looks right at 2ab0df6, but the device-facing route has no live evidence yet. I need that before this can merge.

The only on-device evidence is the CI XCTests at c47fcd5. They call handleAlert in-process on an in-app alert fixture. They skip the CLI, daemon, actOnAppleAlert and runner HTTP route. They also skip the real system ATT prompt that #3627 reports (RunnerTests+Alert.swift#L66). So two things are unproven. First, on a real ATT prompt, alert dismiss should tap "Ask App Not to Track" and iOS should record a denial. Second, a refusal should reach the CLI as error.code ALERT_BUTTON_NOT_FOUND with details.dispatched no and the runner hint, with no retry or relabel in actOnAppleAlert. Please post a live iOS simulator run at this head using the issue's steps: simctl privacy reset all, open --relaunch, alert wait, alert dismiss. The command output must name the "Ask App Not to Track" button, and the TCC.db query must then show auth_value 0. Please also add one alert dismiss --json against an alert with no listed decline, such as the runner's --agent-device-alert-affirmative-only fixture. Its output must show ALERT_BUTTON_NOT_FOUND and details.dispatched no, and a following alert get must still show the alert.

Not blocking, and fine to take or leave: add one Unreleased line to CHANGELOG.md for the dismiss refusal and the new code, since a failed accept with no fitting button now returns ALERT_BUTTON_NOT_FOUND instead of COMMAND_FAILED; and consider a follow-up issue for the macOS helper, whose dismiss still falls back to buttons.last and keeps its own decline list without "ask app not to track", so the #3627 rule holds only on the XCUITest runner.

Of the open inline threads, the two from cubic-dev-ai no longer apply, so please resolve them: RunnerAlertButtonChoice.swift comment is fixed because the header now explains why a positional guess is unsafe, and alert.ts comment is fixed because the help text now names the "dismiss popup" marker and the accept refusal.

Smoke Tests is still queued, not failed, so CI has not given a result yet. No conflicts are known. I read the code and did not run the XCTests or the CLI. The iOS lane result and the affected-check receipt are quoted from the author. Before merge, the live ATT run with auth_value 0 and the CLI refusal showing ALERT_BUTTON_NOT_FOUND with dispatched no must be posted.

@thymikee

thymikee commented Oct 16, 2026 •

Copy link
Copy Markdown
Member Author

Live simulator evidence at this head (2ab0df6d0), through the repo-built CLI, daemon, and runner (not the in-process XCTests).

Device: iPhone 17e simulator, iOS 27. Runner built from this checkout into a lane-owned derived path.

Real system ATT prompt → decline tapped, iOS recorded a denial

Precondition without any global reset: I installed a disposable canary app under its own bundle id with NSUserTrackingUsageDescription, calling ATTrackingManager.requestTrackingAuthorization once its scene is foreground. A fresh install is the issue's precondition — kTCCServiceUserTracking had no row for it (notDetermined), and I touched no other app's or device's privacy state.

open com.callstack.agentdevice.att3627 --platform ios --udid F25C7515-… --relaunch --json   → Opened
alert wait 30000 --json
  "items": ["Ask App Not to Track", "Allow"],
  "message": "Allow “ATTCanary3627” to track your activity across other companies’ apps and websites?"
alert dismiss --json
  "success": true, "message": "dismissed"

Runner activation, one tap only:

AGENT_DEVICE_RUNNER_ALERT_ACTIVATION action=dismiss label=Ask App Not to Track frame=(51.0,509.0,288.0,48.0)

Ground truth after that tap — the canary simulator's TCC.db, own bundle only:

kTCCServiceUserTracking|com.callstack.agentdevice.att3627|0        # auth_value 0 = denied

and the app's own callback logged authorization status=2 (denied).

Refusal reaches the CLI with no dispatch

The runner's own fixture, launched through the CLI:

open com.callstack.agentdevice.runner --platform ios --udid F25C7515-… --relaunch --launch-args --agent-device-alert-affirmative-only
alert get --json → "“Runner” Would Like to Track Your Activity", items ["Allow","OK"]

alert dismiss --json against it:

"code": "ALERT_BUTTON_NOT_FOUND",
"hint": "No alert button was activated. Inspect the current alert before deciding on an action.",
"details": { "runnerErrorCode": "ALERT_BUTTON_NOT_FOUND", "dispatched": "no" }      # exit 1

The runner hint survives actOnAppleAlert → HTTP → CLI unchanged, dispatched is no, and nothing retried or relabeled. The following alert get still returns items: ["Allow","OK"], and the runner logged zero activations for that attempt.

One honest note: alert dismiss prints dismissed rather than echoing the label — the label appears in the alert wait items and in the activation log above. If you want the tapped label in the dismiss payload itself, say so and I'll add it.

I left the CHANGELOG line and the macOS helper follow-up alone as non-blocking, and the two cubic-dev-ai threads are already resolved. Gate receipt and isolated review remain those for 2ab0df6d0; this run added no code change.

@thymikee

Copy link
Copy Markdown
Member Author

Thanks, this live run at 2ab0df6 covers what I asked for. On the real ATT prompt, alert dismiss tapped "Ask App Not to Track" once and TCC.db then showed auth_value 0 for the canary app. On the affirmative-only fixture, the CLI got ALERT_BUTTON_NOT_FOUND with details.dispatched no and the runner hint, and the alert was still there afterward. A fresh install of a separate canary app is a fair stand-in for the simctl privacy reset all step, and it keeps other apps' privacy state untouched.

This is ready for human review. Echoing the tapped label in the alert dismiss payload would be nice, but it is not needed here.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 16, 2026
…ative one

chooseAlertButton answered 'dismiss' with the last button whenever no known
decline label matched. The App Tracking Transparency prompt lists 'Allow'
last and words its decline in a label the list never carried, so
'alert dismiss' reported 'dismissed' while iOS recorded an authorization
(#3627).

Move the accept/dismiss choice into RunnerAlertButtonChoice so the rule is
unit-testable without a screen, and make dismiss answer only a button the
accept rule does not claim: the known decline label when present, otherwise
the last button no accept label matches, which covers declines worded in a
language or phrasing the list does not carry. A prompt whose every button
reads affirmative now fails with 'alert dismiss button not found' and taps
nothing. Recognize 'Ask App Not to Track' as a decline label.

The runner fixture app gains an ATT-shaped prompt with an answer canary so
the iOS lane asserts which button the app really took.
Cold-review follow-ups on the decline-choice rule:

- The new 'dismiss has no button to name' refusal now answers with the
  runner code ALERT_BUTTON_NOT_FOUND and a hint, bound in
  RUNNER_ERROR_CODE_DISPATCH as dispatched: no like every other reply
  that ran nothing, with a row and driver in the dispatch-disclosure
  golden table. Before, the codeless refusal read as dispatched: unknown
  and taught callers to key on the message text.
- The decline-choice fixture carries the one-shot guard its sibling uses
  and an affirmative-only variant, so the refusal's response shape is
  asserted end to end: code ALERT_BUTTON_NOT_FOUND, canary Unanswered.
- The alert command description and the commands.md note state the
  refusal by its typed code, name the one-button 'OK' case, and stop
  promising that the fallback finds a decline.
Cold-review follow-up on the decline-choice rule:

- iOS localizes the affirmative button too, so a fully localized prompt
  leaves every label unclaimed and the last-unclaimed fallback answered
  the localized 'Allow' — the same authorize-on-guess failure #3627 is
  about, one locale over. The fallback now fires only when exactly one
  button is unclaimed; when more than one is, the choice refuses with
  ALERT_BUTTON_NOT_FOUND and taps nothing, like an all-affirmative
  prompt. The localization unit vector carries both buttons in the
  locale, not a German decline paired with an English 'Allow'.
- 'ask app not to track' in the decline list is pinned by the shape only
  it decides: ['Ask App Not to Track', 'Learn More'].
- The no-button-activated hint is named once and shared by the refusal
  and the unactivated verification branches so the two cannot drift.
- The command description and commands.md state the scoped guarantee
  and the runner boundary instead of promising any unclaimed label is a
  decline, and the two behavior pins are annotated as pins.
Second cold-review round on the decline-choice rule:

- 'Accept', 'I Agree', 'Got it', 'Allow Tracking' and 'Use App' were
  outside isAcceptButton, so the sole-unclaimed fallback activated a
  one-button consent prompt of that copy and reported 'dismissed' —
  the incident's shape one label away from the list. The accept list
  now claims them, with pins.
- The two canary fixtures resolve under the sibling lane's resolution
  allowance instead of the 10s command timeout: resolution is a pure
  read that a contended hosted simulator has spent 40s on, and timing
  it out would read as a red fix.
- commands.md and the CLI description state the guarantee as the
  runner's English label lists actually deliver, attribute the ATT
  decline to its list entry, and stop promising 'every button reads
  affirmative' beyond what the lists recognize.
- Pin comments state the invariant they hold rather than this PR's
  review history.
Parent-evidenced vectors showed the sole-unclaimed fallback still
reached consent copy the accept list had not been taught yet: 'Accept
all', 'Agree to Terms', 'I accept', 'Accept terms' were each tapped and
reported 'dismissed'. Growing the accept list cannot close an
open-ended copy space, so the rule stops guessing:

- dismiss resolves only a label isDismissButton recognizes; any other
  prompt — unknown wording, localization, consent copy — answers
  ALERT_BUTTON_NOT_FOUND and taps nothing, the issue's stated
  alternative to picking the decline. Unknown buttons stay reachable by
  name through 'alert get' plus snapshot + press.
- With no fallback, the accept list needs no consent-label arms race
  and returns to its base membership.
- The pure tests pin the refusal set with the parent's exact shapes,
  keep the known-decline positives including the 'Ask App Not to Track'
  entry pin, and keep accept's guards unchanged.
- commands.md, the CLI description, and the code contract constant's
  doc say the list-scoped rule and name the press-by-label path.
…ts marker

alert dismiss on an app-owned popover resolved through the native
"dismiss popup" marker regressed: that surface's close control ("Close
popover") is on no decline list, and base reached it through the
positional fallback. Restore it as a property of the surface, not a
position: only a RunnerAlert whose source is the marker window may take
its last actionable element, so consent prompts (never marker-marked)
keep the #3627 refusal. Pin the surface through chooseAlertButton and
the label rule in the pure table.

Also from review: scope the decline-list claim to the Apple XCUITest
runner in the CLI description and wire-code contract, declare
ALERT_BUTTON_NOT_FOUND in RunnerWireErrorCode where the protocol names
wire codes, and drop the migration narration from the user doc.
@thymikee
thymikee force-pushed the fix/alert-decline-choice branch from 2ab0df6 to 9203a7a Compare October 16, 2026 17:21
@thymikee
thymikee merged commit 406a977 into main Oct 16, 2026
20 checks passed
@thymikee
thymikee deleted the fix/alert-decline-choice branch October 16, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iOS: alert dismiss taps "Allow" on the App Tracking Transparency prompt

1 participant