Skip to content

feat(webkit): roll to r2362 - #42736

Open
microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-webkit/2362
Open

microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-webkit/2362

Conversation

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

Browser build: https://github.com/microsoft/playwright-browsers/commit/a02e9f369ffd6f175d166d5a91850aa30c660746. Build has full platform coverage - check that all tests pass there.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

3 failed
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://playwright.download.prss.microsoft.com/dbazure/download/playwright) @package-installations-macos-latest
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-ubuntu-22.04-node20

6 flaky ⚠️ [chromium-library] › library/video.spec.ts:356 › screencast › should work for popups `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-event-request.spec.ts:181 › should return response body when Cross-Origin-Opener-Policy is set `@firefox-ubuntu-22.04-node20`

51719 passed, 1247 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate @mcp-windows-latest-firefox

8563 passed, 1446 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🔴 The CDN failures are real — WebKit r2362 isn't on the ESRP mirror yet

Hi, I'm the Playwright bot and I took a look at the failing tests.

Two of the three playwright cdn failover variants fail because webkit/2362 is missing from the ESRP CDN — https://playwright.download.prss.microsoft.com/dbazure/download/playwright/builds/webkit/2362/webkit-mac-26-arm64.zip returns 400 right now, while 2360 returns 200 and plain https://cdn.playwright.dev/builds/webkit/2362/... returns 200. That's why the third variant (https://cdn.playwright.dev) passed. Nothing to fix in the diff — re-run once the build propagates to ESRP. The Firefox failure is pre-existing.

Details

This PR only bumps the WebKit revision in browsers.json, so the blast radius is browser download and WebKit behaviour. No WebKit test failed; the noise is one broken mirror plus one already-red Firefox test.

Caused by this PR

  • [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) and the https://playwright.download.prss.microsoft.com/... variant, both on package-installations-macos-latest — the test pins one CDN as the only working host, so it can't fall back. Chromium downloads fine, then FAILED installation WebKit 26.6 (playwright webkit v2362). Both ESRP hosts 404/400 for the new revision; cdn.playwright.dev serves it. Failed all 4 attempts in the run, so it isn't a hiccup. Same test is otherwise green in 930 of 931 runs in the CI results DB. Not a code problem — the artifact just hasn't landed on the ESRP mirror.

Pre-existing flake / infra

  • [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open — broken on Firefox independently of this PR: 38 of 38 runs failed, including main-branch pushes (e.g. sha 654654fc, run 34864024203) and other PRs (fix(chromium): bypass service workers on the storage state page #42664, sha 68d238eb). Always the same expect(received).toEqual(expected) diff with 4 unexpected cookies. The same test passes on chromium and webkit. A WebKit revision bump can't reach it.
  • The 6 flaky entries all passed on retry and sit in chromium/firefox code this PR doesn't touch.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

🔴 The installation failures are on this roll — WebKit r2362 isn't on the download mirror yet

Hi, I'm the Playwright bot and I triaged the failing CI here.

npx playwright install fails on macOS because webkit-mac-26-arm64.zip for r2362 returns 400 GatewayExceptionResponse from playwright.download.prss.microsoft.com. r2361 and r2360 serve 200 from the same host, and cdn.playwright.dev already serves r2362 — so this is the mirror lagging behind the new build, not a code defect. Re-run the installation job once the build has propagated. The other two failures are pre-existing and unrelated.

Details

The PR only bumps webkit in packages/playwright-core/browsers.json, so nothing in it can reach Firefox or Chromium product code. Two of the three failure groups are on Firefox and have history that predates this branch.

Caused by this PR

  • [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (both the cdn.playwright.dev and playwright.download.prss.microsoft.com variants, @package-installations-macos-latest) — the install downloads Chromium and Firefox fine, then dies on builds/webkit/2362/webkit-mac-26-arm64.zip with response status code: 400, body GatewayExceptionResponse, after all 5 attempts. Checked by hand just now:

    URL status
    prss…/builds/webkit/2361/webkit-mac-26-arm64.zip 200
    prss…/builds/webkit/2362/webkit-mac-26-arm64.zip 400
    cdn.playwright.dev/…/builds/webkit/2362/webkit-mac-26-arm64.zip 307

    So the revision this PR points at isn't mirrored yet. The test is otherwise solid — 1 failure in 632 runs in the results DB, and that one is this PR. No source change needed; this should go green on a re-run after the mirror catches up.

Pre-existing flake / infra

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 2"

5 failed
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @tracing-firefox
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-macos-15-large
❌ [firefox-library] › library/browsertype-connect.spec.ts:288 › launchServer › should send default User-Agent and X-Playwright-Browser headers with connect request @firefox-macos-15-large
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-windows-latest
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-macos-15-xlarge

30 flaky ⚠️ [chromium-library] › library/browsercontext-page-event.spec.ts:173 › should work with Ctrl-clicking `@tracing-chromium`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:373 › should produce network events, routing, and annotations for Service Worker `@tracing-chromium`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:213 › should intercept service worker requests (main and within) `@chrome-ubuntu-22.04`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:373 › should produce network events, routing, and annotations for Service Worker `@chrome-ubuntu-22.04`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@chromium-macos-15-large`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@chromium-macos-14-xlarge`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@chrome-beta-ubuntu-22.04`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:373 › should produce network events, routing, and annotations for Service Worker `@chrome-beta-ubuntu-22.04`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:190 › selector generator › should escape text with slash `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:297 › selector generator › should use internal:has-text with regexp `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:516 › selector generator › should work with tricky attributes `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:633 › selector generator › should generate exact role when necessary `@chrome-macos-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@tracing-firefox`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@tracing-firefox`
⚠️ [firefox-page] › page/page-goto.spec.ts:90 › should work with Cross-Origin-Opener-Policy `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-macos-15-large`
⚠️ [firefox-library] › library/debug-controller.spec.ts:350 › should highlight inside iframe `@firefox-macos-15-large`
⚠️ [firefox-library] › library/trace-viewer.spec.ts:1915 › should not leak recorders `@firefox-macos-15-large`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-macos-15-xlarge`
⚠️ [webkit-library] › library/browsertype-connect.spec.ts:813 › run-server › should upload a folder `@tracing-webkit`
⚠️ [webkit-library] › library/har.spec.ts:472 › should return receive time `@webkit-windows-latest`
⚠️ [webkit-page] › page/locator-query.spec.ts:286 › count() should not throw during navigation `@webkit-windows-latest`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:1915 › should not leak recorders `@webkit-macos-15-large`

111372 passed, 4739 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🔴 One failure is caused by this PR — but it's mirror lag, not a code bug

playwright-cdn.spec.ts:43 fails because webkit r2362 isn't on the download mirror yet. Everything else is pre-existing.

Details

The PR only bumps browsers.json to webkit r2362, so the only CI surface it can touch is webkit downloads — and that's exactly the one failure that's attributable. The four Firefox/MCP failures are all reproducible on main or across dozens of unrelated SHAs.

Caused by this PR

  • [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (both the cdn.playwright.dev and playwright.download.prss.microsoft.com variants, @package-installations-macos-latest) — Chromium and Firefox download fine, then the install dies on builds/webkit/2362/webkit-mac-26-arm64.zip with status 400 / GatewayExceptionResponse after all 5 attempts. Re-checked by hand just now:

    URL status
    prss…/builds/webkit/2361/webkit-mac-26-arm64.zip 200
    prss…/builds/webkit/2362/webkit-mac-26-arm64.zip 400
    cdn.playwright.dev/…/builds/webkit/2362/webkit-mac-26-arm64.zip 307

    So the revision this PR points at isn't mirrored yet. The test itself is solid — 1 failure in 931 runs in the results DB, and that one is this PR. No source change needed; a re-run once the mirror catches up should be green.

Pre-existing flake / infra

  • [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open (4 bots: @tracing-firefox, @firefox-macos-15-large, @firefox-macos-15-xlarge, @firefox-windows-latest, @firefox-ubuntu-22.04-node20) — broken on main, not here. It fails the same 5-of-28 Firefox bots on main pushes cca996e, 9ae5377, a024162, 52a0e72, 4dc79d4, da3553d, 654654f with no PR attached, plus unrelated PRs feat(firefox): roll to r1546 #42735, fix(network): preserve dates in response headers #42734, fix(reporter): do not hang perfetto writes after a stream error #42715, fix(server): end range responses when the file stream errors #42716, Fix spelling of restoring in credentials docs #42718, test: evaluate() with falsy result #42728. It trips on a cookie assertion (fromServiceWorker leaking into storage state) — nothing a webkit revision bump reaches.

  • [firefox-library] › library/browsertype-connect.spec.ts:288 › launchServer › should send default User-Agent and X-Playwright-Browser headers with connect request (@firefox-macos-15-large) — timeout flake confined to the macOS large bots: 5 failures in 2994 runs, all timedOut, the other four on main pushes (3eb0ce9 firefox, a5e5632 / c055448 / 43a9121 chromium). Chromium hitting it rules out a webkit roll as the cause.

  • [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate (@mcp-windows-latest-firefox) — long-running flake: 49 failed runs out of 3572, spread over 85 distinct SHAs and 40 distinct PRs, including main pushes. Unrelated to browser binaries.

Triaged by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants