Skip to content

feat(trace-viewer): show screencast frames during playback - #42751

Open
Pavel Feldman (pavelfeldman) wants to merge 1 commit into
microsoft:mainfrom
pavelfeldman:playback-screencast
Open

Pavel Feldman (pavelfeldman) wants to merge 1 commit into
microsoft:mainfrom
pavelfeldman:playback-screencast

Conversation

@pavelfeldman

@pavelfeldman Pavel Feldman (pavelfeldman) commented Sep 16, 2026

Copy link
Copy Markdown
Member

Summary

  • Show the screencast in the snapshot area while playing and while dragging the scrubber, so that animations are visible.
  • Snap back to the action snapshot when the playback stops, the scrubber is released, an action is selected or hovered, or a locator is picked.
  • Traces with a video play the video itself, the thumbnails only index the film strip lanes.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a look at the CI failures here.

🟢 CI looks clear — the one failure is a known flake

[firefox] › mcp/cli-core.spec.ts:149 › upload multiple files is bimodal on mcp-windows-latest-firefox6 failures in 696 runs, with the same assertion failing on main (sha 52a0e720) and on unrelated PRs #42431, #42628 and #42713. This PR only touches trace-viewer UI code, which the MCP file-upload flow doesn't reach.

Details

Overall: 1 failed, 10 flaky across the two reports. Nothing points at this PR — the diff is confined to packages/trace-viewer/src/ui/* (playbackControl.tsx, snapshotTab.tsx, ariaModeView.tsx, workbench.tsx, snapshotTab.css) plus a new tests/library/trace-viewer-scrub.spec.ts. That new test passed.

Pre-existing flake / infra

  • [firefox] › mcp/cli-core.spec.ts:149 › upload multiple files @mcp-windows-latest-firefox — pre-existing flake. Run history for this test on this bot (oldest first, one square per run attempt):

    🟩🟥🟩🟩🟩🟩🟩🟩🟩🟩🟩🟥🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟥

    It's Windows-only in practice (6/696 there vs 1/706 on Linux and 1/709 on macOS). The failing assertion is the same one every time — the browser_file_upload tool errors out because no file chooser is showing yet, so the expected await fileChooser.setFiles( never lands in the code output. That's a timing race between the click and Firefox surfacing the chooser, unrelated to anything here.

  • The 10 flaky tests in "tests 1" (service workers, third-party cookies, page-set-input-files, codegen upload) all passed on retry and sit in library/page areas the diff doesn't touch.

Triaged by the Playwright bot - agent run

@pavelfeldman
Pavel Feldman (pavelfeldman) force-pushed the playback-screencast branch 2 times, most recently from 467a8ea to 7290921 Compare September 18, 2026 00:13
@pavelfeldman
Pavel Feldman (pavelfeldman) marked this pull request as ready for review September 18, 2026 00:13
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Playback and dragging the scrubber show the screencast at the current
time. Stopping, releasing the scrubber, hovering an action or picking a
locator snap back to the action snapshot.

Traces with a video play the video itself, the thumbnails only index
the film strip lanes.
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

Warning

The triggering workflow run ended with status cancelled. Results below may be incomplete — blob reports from cancelled or timed-out shards are missing, so passing/failing counts do not reflect the full test suite.

615 passed, 160 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

Warning

The triggering workflow run ended with status cancelled. Results below may be incomplete — blob reports from cancelled or timed-out shards are missing, so passing/failing counts do not reflect the full test suite.

1217 passed, 5 skipped


Merge workflow run.

target={ariaModeTargets[snapshotTab]}
point={snapshotTab === 'action' ? action?.point : undefined}
box={snapshotTab === 'action' ? action?.box : undefined}
screencastFrame={screencastFrame}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if video drifts away from the action and the image is different from the snapshot? may be confusing for the user that stopped on an action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants