Skip to content

feat(webkit): roll to r2363 - #42737

Closed
microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-webkit/2363
Closed

microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
roll-into-pw-webkit/2363

Conversation

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

Browser build: https://github.com/microsoft/playwright-browsers/commit/391436975620832e133460f1d282db61d4973704. Build has full platform coverage - check that all tests pass there.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

8564 passed, 1446 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

3 failed
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://playwright.download.prss.microsoft.com/dbazure/download/playwright) @package-installations-macos-latest
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-ubuntu-22.04-node20

5 flaky ⚠️ [chromium-library] › library/video.spec.ts:699 › screencast › should capture full viewport on hidpi `@chromium-ubuntu-22.04-arm-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-goto.spec.ts:90 › should work with Cross-Origin-Opener-Policy `@firefox-ubuntu-22.04-node20`

51720 passed, 1247 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🔴 The CDN failover failures are about this roll; the Firefox one is broken on main

Hi, I'm the Playwright bot and I triaged the CI failures on bef5723.

playwright-cdn.spec.ts:43 fails while downloading webkit v2363 for mac-26-arm64 from the two ESRP endpoints — the revision this PR introduces. The Firefox storage-state failure is unrelated: it has failed 100% of the time on main since Sep 10.

Details

Caused by this PR

  • [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) and the https://playwright.download.prss.microsoft.com/dbazure/download/playwright variant, both @package-installations-macos-latest. Chromium, headless shell, ffmpeg and firefox download fine; the run ends with

    pw:install -- from url: https://…/dbazure/download/playwright/builds/webkit/2363/webkit-mac-26-arm64.zip
    pw:install attempt #5 - ERROR: Download failure, code=1
    pw:install FAILED installation WebKit 26.6 (playwright webkit v2363) with error: Error: Download failure, code=1
    

    The third variant of the test, plain https://cdn.playwright.dev, passed — so webkit-mac-26-arm64 for r2363 is served by cdn.playwright.dev but not (yet) by the dbazure ESRP paths. That looks like publication/propagation of the new build rather than a defect in the one-line revision bump, so a re-run once the ESRP mirror has the file is the thing to try. Worth confirming that the mac-26-arm64 artifact really did publish to ESRP, since the same two tests failed the same way on the previous roll, feat(webkit): roll to r2362 #42736 (webkit v2362, also mac-26-arm64).

    Across the results DB these two tests are otherwise solid: 2 failing runs out of 635, and both are webkit roll PRs (this one and feat(webkit): roll to r2362 #42736). Nothing else in the recent window hits them.

Pre-existing

  • [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open — broken on main, not a flake. It has failed every single run since 2026-09-10: 0 passes out of 31 runs on firefox-ubuntu-22.04-node20, plus firefox-macos-15-large/-xlarge, firefox-windows-latest, tracing-firefox, moz-firefox-nightly-library and bidi-chromium-library — 32 distinct SHAs across 18 PRs. Always the same assertion:

    expect(received).toEqual(expected)
    + Object { "name": "fromServiceWorker", "value": "yes" }
    

    A webkit revision bump can't reach a Firefox service-worker cookie assertion.

The 5 flaky entries in the report (video hidpi, third-party cookies, emulate-media, COOP) all passed on retry and are Chromium/Firefox-only, so they're out of scope for a webkit roll.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 2"

1 fatal errors, not part of any test
6 failed
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @tracing-firefox
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-macos-15-large
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-windows-latest
❌ [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open @firefox-macos-15-xlarge
❌ [android-native] › android/device.spec.ts:56 › androidDevice.fill
❌ [android-native] › android/webview.spec.ts:44 › should navigate page externally

29 flaky ⚠️ [electron-api] › electron/electron-app.spec.ts:243 › should test app that opens window fast `@electron-ubuntu-latest`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@chrome-ubuntu-22.04`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:436 › should produce network events, routing, and annotations for Service Worker (advanced) `@chromium-macos-15-large`
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:105 › cli codegen › should upload a single file `@chromium-macos-15-large`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:213 › should intercept service worker requests (main and within) `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/chromium/oopif.spec.ts:179 › should take screenshot `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:213 › should intercept service worker requests (main and within) `@chrome-beta-ubuntu-22.04`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:373 › should produce network events, routing, and annotations for Service Worker `@chrome-beta-ubuntu-22.04`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:179 › serviceWorker(), and fromServiceWorker() work `@chrome-macos-latest`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:301 › should report intercepted service worker requests in HAR `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:160 › selector generator › should try to improve text `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:272 › selector generator › should use readable id `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:426 › selector generator › should prioritize attributes correctly › placeholder `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:430 › selector generator › should prioritize attributes correctly › name `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:683 › selector generator › should generate noText: name from aria-label `@chrome-macos-latest`
⚠️ [chromium-library] › library/trace-viewer-scrub.spec.ts:82 › should stop and reset to first action `@chrome-macos-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@tracing-firefox`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/browsercontext-viewport.spec.ts:29 › should set the proper viewport size `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/permissions.spec.ts:296 › local network request is allowed from public origin `@firefox-macos-15-xlarge`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:121 › should show tracing.group in the action list with location `@webkit-macos-15-xlarge`
⚠️ [webkit-library] › library/screencast.spec.ts:57 › applies backpressure while async onFrame callback is pending `@webkit-macos-15-large`

111372 passed, 4739 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟢 None of the failures are caused by this roll

Hi, I'm the Playwright bot and I triaged the failing CI here.

This PR bumps WebKit to r2363 and touches nothing but browsers.json — yet zero WebKit tests failed. All 6 failures are Firefox and Android, and both stories are reproducible on SHAs that predate this branch.

Details

Pre-existing flake / infra

  • [firefox-library] › library/browsercontext-storage-state.spec.ts:453 › should work when service worker is intefering and the origin is not open (@tracing-firefox, @firefox-macos-15-large, @firefox-windows-latest, @firefox-macos-15-xlarge) — not a flake, a pre-existing break. In the aggregated CI results this test has failed 46 of 46 runs across 30 distinct SHAs and 17 different PRs since 2026-09-10 — 42654, 42684, 42707, 42709, 42716, 42726, 42728, 42736 among them, plus pushes to main. Never passed once in that window. Same assertion every time (expect(received).toEqual(expected), 4 unexpected cookies received). It also fails on moz-firefox-nightly-library and bidi-chromium-library, which a WebKit binary can't reach.

  • [android-native] › android/device.spec.ts:56 › androidDevice.fill and [android-native] › android/webview.spec.ts:44 › should navigate page externally — genuinely bimodal: 12 failed / 332 passed runs, spread over 11 SHAs and PRs 42101, 42441, 42523 plus main pushes, going back to 2026-08-03. Both fail with java.lang.RuntimeException: Timed out waiting for selector — the usual emulator-flake signature. The Android bots run Chromium on a device; the WebKit revision in browsers.json isn't downloaded or used there.

The 29 flaky results are likewise all Chromium/Electron/Firefox service-worker and codegen regulars, plus two WebKit ones that passed on retry.

One caveat on my own confidence: the report also lists 1 fatal error not attached to any test, which I couldn't attribute from the aggregated results. It didn't produce a test failure, so it doesn't change the verdict, but the merge workflow run has the raw log if someone wants to confirm.

Triaged by the Playwright bot - agent run

@dcrousso

Copy link
Copy Markdown
Contributor

included in #42748

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant