Skip to content

devops(bidi): publish BiDi reports to a git branch instead of Azure - #42760

Merged
Yury Semikhatsky (yury-s) merged 2 commits into
microsoft:mainfrom
yury-s:bidi-reports-branch
Sep 17, 2026
Merged

Yury Semikhatsky (yury-s) merged 2 commits into
microsoft:mainfrom
yury-s:bidi-reports-branch

Conversation

@yury-s

@yury-s Yury Semikhatsky (yury-s) commented Sep 17, 2026

Copy link
Copy Markdown
Member

Summary

  • Corp policy no longer allows uploading reports to mspwblobreport blob storage.
  • Publish the per-channel csv reports to the orphan bidi-reports branch instead, where GitHub renders them as tables and the commit history gives day-over-day diffs. The branch is created on the first run, so no manual setup is needed.
  • Publishing runs in a separate job after the matrix, so only that job needs contents: write.
  • Also add the report to the job summary.

Report urls once it lands:

The AZURE_BLOB_REPORTS_* repo secrets are now unused and can be deleted.

Fixes #42759

Azure blob storage is no longer available for report hosting. Publish the
per-channel csv reports to the orphan bidi-reports branch instead, where
GitHub renders them as tables and the commit history gives day-over-day
diffs. Also emit a markdown copy of the report into the job summary, and
drop the flakiness-results environment left dead by a024162.
Publishing from each matrix job meant two jobs racing to push to one
branch, which needed a retry loop and gave the whole test job push
rights. Publish once after the matrix instead, from the csv artifacts
the test jobs already upload: no race, one commit per run, and the test
job keeps contents:read.

Also make the reporter's markdown output an explicit option rather than
deriving it from the csv path, so local runs no longer write a stray
report.md.
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [firefox] › mcp/idle-timeout.spec.ts:19 › closes the browser after the idle timeout and relaunches it on the next call @mcp-windows-latest-firefox

8589 passed, 1446 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟢 CI is clear — the one failure is a known pre-existing flake

Hi, I'm the Playwright bot and I triaged the failing CI run here.

The only failure is [firefox] › mcp/idle-timeout.spec.ts:19 on Windows, and the exact same test with the exact same error failed on PR #42718 two days ago. This PR only touches the BiDi workflow and its CSV reporter, so it can't reach the MCP idle-timeout path.

Details

Overall: 1 failed, 8589 passed, 1446 skipped. Nothing in this PR's diff (​.github/workflows/tests_bidi.yml, tests/bidi/csvReporter.ts, tests/bidi/playwright.config.ts) is loaded by the MCP suite — the BiDi config and reporter are only used by the tests_bidi.yml workflow, which isn't the one that failed.

Pre-existing flake / infra

  • [firefox] › mcp/idle-timeout.spec.ts:19 › closes the browser after the idle timeout and relaunches it on the next call (mcp-windows-latest-firefox) — in the aggregated CI results this test has failed 2 of 102 runs (2%) on this bot and passed the other 100; it is green on all 12 other MCP bots (0 failures in ~1300 runs). The other failure is run 34915545227 on PR Fix spelling of restoring in credentials docs #42718 (sha d979f021), which has nothing to do with this change.

    Both failures share one signature — Firefox on Windows tears down early and the snapshot comes back undefined:

    - ObjectContaining { "snapshot": StringContaining "Hello, world!" }
    + Object { "snapshot": undefined }
    
    Error: browserBackend.callTool: Target page, context or browser has been closed
    

    The test asserts the browser is relaunched after the idle timer fires, so a slow Firefox launch on a loaded Windows runner races the timeout. A retry should be green.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 failed
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://playwright.download.prss.microsoft.com/dbazure/download/playwright) @package-installations-macos-latest

7 flaky ⚠️ [chromium-library] › library/video.spec.ts:521 › screencast › should capture static page in persistent context Radoslav Kirilov (@smoke) `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:690 › screencast › should capture full viewport `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:690 › screencast › should capture full viewport `@realtime-time-library-chromium-linux`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-ubuntu-22.04-node20`

51971 passed, 1247 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟢 CI is clear — both failures are a CDN outage, not this PR

Hi, I'm the Playwright bot and I triaged the failing CI run here.

Both failures are playwright-cdn.spec.ts:43 on package-installations-macos-latest, and they are failing right now on main too — including on commit 8bb2c097, which is this PR's own base. The CDN returns HTTP 400 for the WebKit mac-arm64 build; nothing in this diff touches installation or downloads.

Details

Overall: 2 failed, 7 flaky, 51971 passed, 1247 skipped. The diff is .github/workflows/tests_bidi.yml, tests/bidi/csvReporter.ts and tests/bidi/playwright.config.ts — the BiDi workflow and its CSV reporter. None of that is loaded by the installation tests, which run npx playwright install against the real CDN.

Pre-existing flake / infra

Also seen

  • The 7 flaky entries (video screencast, third-party cookies, beforeunload, emulate-media) all passed on retry and are in areas this PR doesn't touch.

Triaged by the Playwright bot - agent run

@yury-s
Yury Semikhatsky (yury-s) merged commit 6391967 into microsoft:main Sep 17, 2026
44 of 47 checks passed
@yury-s
Yury Semikhatsky (yury-s) deleted the bidi-reports-branch branch September 17, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DevOps] Migrate BiDi test report from blobstorage

2 participants