Repository navigation
Conversation
Streamed browser tabs (web, remote desktop, mobile) were a JPEG canvas plus a hidden input, so screen readers found one text field and nothing else. Viewers can now turn on an accessible view. The server enables Chromium's Accessibility domain for that tab only while someone subscribes, joins the per-frame trees (including out-of-process iframes), drops ignored wrappers while keeping their descendants, hides secret field values, and sends full coalesced snapshots with backpressure redelivery. Read-only viewers may subscribe; activate and set-value actions run only for the controlling viewer under SessionControl and only against the current document and node ids. A shared client-runtime renderer draws the snapshot as native headings, landmarks, links, buttons and form fields, updating in place by node id. Web adds a toggle to the streamed view's bar; mobile adds one to the full-screen viewer and renders inside the same WebView. Passive floating previews never subscribe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntact - Editable field values are sent exactly (whitespace and label-equal text kept), still bounded and still withheld for secret or masked fields. - Tree items render as role=treeitem containers so nested items stay reachable and activatable; Enter and Space activate them. - Child sync removes obsolete nodes first and keeps the focused child in place while siblings move around it. - Out-of-process frame sessions enable the Accessibility domain and listen for changes; their listeners, domain and session are released on frame navigation, detach and unsubscribe. - Any frame navigation or detach clears action targets until the next capture, and a capture that spans a frame change cannot be acted on. - A viewer backlog reset keeps the newest queued accessibility document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
added
vouch:trusted
|
Note Written by updated the original branch at thanks @shivamhwp for the shared semantic renderer, opt-in snapshots, and existing control checks. those remain intact. i scoped the mobile hidden-input styles, kept its overlay inside the viewport, disabled the raw keyboard buffer while accessible view is open, and made the web toggle local to each surface. real chromium also exposed lost inline whitespace and secret values in field descendants; both now have failing-first regressions and fixes. unread iframe content now produces an incomplete-document notice. verification passed:
unverified: native ios/android webviews, electron, and an actual screen reader. theo's requested approval is still outstanding, so this remains a draft. @shivamhwp: the desktop cdp relay currently rejects separate iframe targets; unsupported embedded content is now reported, but complete desktop out-of-process iframe support remains a relay follow-up. the main/updated screenshots use the same w3c page and viewport: the mobile document comparison uses the same streamed form data. the before capture reapplies the original pr's global textarea rule; the after capture uses the scoped rule: video playback and seeking passed at the uploaded url. git.cafe currently displays external videos as links; the four screenshots are native attachments. |
|
Note Written by Verified in a real dev environmentverified unchanged head
short runtime transcript: the screenshots use the same W3C page and 1440 by 1000 client viewport. the 15-second recording shows the actual form, pixel/semantic switching, release/take control, multiline editing and keyboard exit. playback is slowed to half speed for inspection. the exported clip and uploaded player were inspected; playback and seeking passed. Not verifiednative iOS/Android WebViews, Electron/remote desktop, macOS/Windows, an actual screen-reader session, multi-device/relay connections, and out-of-process iframe behavior were not exercised here. this pass does not resolve the PR's existing screen-reader/native-client verification caveats or its requested approval. startup note: the root checkout initially selected |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unproven: actual screen-reader navigation and forms, native iOS/Android WebViews, and packaged Linux/macOS Electron. The desktop CDP relay rejects separate iframe targets, so some embedded content remains unavailable; Accessible view reports incomplete documents. This PR stays draft pending verification.
Approval required: this PR touches Metro bundling configuration, scroll containers, and the wire protocol. It requires Theo's approving GitHub review before merge.
Ready to merge: no. Evidence findings R1, R2 and R3 remain open, and Theo has not approved.
Streamed browser tabs expose pixels and a keyboard buffer, so screen readers cannot read the remote page. Add an opt-in Accessible view to web, remote desktop, and full-screen mobile streams. A shared renderer exposes text, headings, links, and common form controls.
The server captures full accessibility snapshots only while viewers subscribe. Read-only viewers can read them; actions retain permission and control checks. Navigation invalidates action targets, interrupted edits stop before further mutations, and updates preserve local drafts. Protected fields omit values and descendants. Text keeps inline spaces, and inaccessible frames produce an incomplete-document notice.
The mobile raw-input styles apply only to the keyboard buffer, so multiline semantic fields remain visible. Accessible mode disables that buffer to let keyboard navigation leave the document controls. Web toggles are local to each browser surface.
Refs #25271. An actual screen-reader workflow is still unverified.
Validation at head
63c8128b63b2482dcc5c749c6aaeb08a3c7443c6, with maine5559038165337caaf84ec162e3343b6c25120da:Merged main and mechanically retained both mobile React test registrations: accessible-document and workspace-tab state. The sole manual resolution removes four conflict-marker lines.
Fresh independent GPT-6.1-sol review of
1704b217a861853b7e3c22780524bd7fb5a756cbagainst maine5559038165337caaf84ec162e3343b6c25120dafound no new findings. Accessibility subscription lifetimes, current-document guards, control checks, backlog handling, the web toggle, and hidden-input behavior remain intact.The review confirmed that main's capacity reservation, file-picker cleanup on page replacement, stale-download cleanup, serialized profile clearing, and typed viewport dimensions fixes remain preserved. Both mobile tests are registered in the correct React project.
M1 remains fixed. Previous raw-log evidence retains its original commit attribution; the fresh execution below covers this merged head. R1, R2 and R3 remain open.
V1, found by execution after the source review, is fixed in
63c8128b63b2482dcc5c749c6aaeb08a3c7443c6: the new ServerBrowserSurface browser-test stream mock now includes the accessibility method used during mount. This follow-up changes only the test fixture, with no production behavior change and no new test case.Initial merge job
j_2g2erzrylp2roywyvpbaon1704b217a861853b7e3c22780524bd7fb5a756cbfailed its changed-file browser tests because the mock lacked client.accessibility. Its separate 153-test run passed, but that job was not a pass overall. The final-head rerun below supersedes it.Fresh Linux Boat job
j_b7o2prikwgf7gue6qly7, stagetypecheck-ax-mobile-final, large worker, VMbx_fu2csfun, attempt 1, using thet3codetemplate. The job finished with exit 0. Node.js 26.12.1 andvp i --frozen-lockfile.vp run check:changed e5559038165337caaf84ec162e3343b6c25120dapassed, including 39 package typechecks, 103 tests across eight files, and 10 selected React compiler checks. Complete commands and results are in the execution log below.A separate focused
vp test runcovered ServerBrowser, accessibility, stream, context/page ownership, shared client renderer/stream, the mobile accessible document, and workspace-tab state. All 153 tests across nine files passed.GitHub CI for this pushed head is tracked separately in the PR checks. Runs on earlier commits remain historical.
Evidence review:
50a12e6daa352baf52f416ca2a235494a5b0c0f6. They are historical and do not verify this merged build. They used no screen reader, native WebView, packaged Electron, or relay.Native screen-reader verification remains blocked by device tooling. AgentDevice refused the VoiceOver row as covered by a toolbar, including the retry after screenshot inspection. The alternative VoiceOver Settings deep link also failed twice, with iOS error 115 on the advised debug retry. The 2026-10-31 blocker recheck used a fresh Simlock iPhone 17e lease on iOS 27.0. Settings opened, but AgentDevice again refused the VoiceOver row as covered by toolbar "Toolbar" at (0, 0 390x844), diagnostic mvvvw2uq-6fe3786b. The screenshot showed VoiceOver Off. The fresh lease was released with device_close and shutdown=true. No native T3 workflow was verified. Theo's approving GitHub review remains missing and the PR remains draft.
Complete execution log for job j_b7o2prikwgf7gue6qly7
ANSI terminal control sequences removed. Original execution.log SHA-256:
c366b13a4944e65f701bf93d83e9c2c2086f0e504963fcfe50f98d8b03927556.Maria's historical
50a12e6web screenshots show the same W3C page and 1440 by 1000 viewport:The earlier mobile document comparison uses the same form data and compares the original global textarea rule with the scoped rule. This was a generated document in Chrome, not a native WebView:
Implemented with Claude Opus 5.5 in Claude Code and GPT-6.1-sol in Codex. Reviewed with GPT-6.1-sol in Codex. Maria contributed fixes and browser verification with GPT-6.1-sol.