Show failure details in GitHub Actions step-summary collapsible sections by azat-msft · Pull Request #10633 · microsoft/testfx

Azat Mukhametshin (azat-msft) · 2026-08-18T10:15:45Z

What

The GitHub Actions step summary previously listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

Namespace.TestClass.TestMethod — 2.40s

Exception: System.InvalidOperationException

Location: src/Calc.cs:42

Expected: 42
Actual:   41

   at Calc.Add() in Calc.cs:line 42

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

--report-gh-failure-details on|off (default on) restores the previous compact list.

Bounding the output

Jakub Jareš (@nohwnd)'s point about the job-summary size limit drove most of the design. GitHub caps a summary at 1 MiB and drops it entirely when exceeded — it does not truncate — so the bounds have to hold in every shape, and every reduction is stated in the rendered output rather than applied silently.

Bound	Limit	On overflow
Message length	2,000 chars	clipped, `[... truncated]` appended
Message rows	30 lines	clipped, `[... truncated]` appended
Stack trace length	4,000 chars	clipped, `[... truncated]` appended
Stack trace rows	30 frames	clipped, `[... truncated]` appended
Failure list	20 per project	`Showing the first 20 of N failed tests`
Expanded detail	shared budget	remaining failures degrade to compact lines + a note counting them
Whole project section	shared budget	section condenses to a one-line verdict that says why

Two of these came directly out of validation rather than design:

Row limits. A character cap alone does not bound readability — a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable.

A shared budget, not a per-section one. The cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. The aggregate path divides the budget across modules; the direct path measures what sibling projects already wrote and claims only the remainder. Per-project overhead is reserved before dividing, so the bound applies to the rendered file rather than to the diagnostics alone.

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the aggregation fragment written to disk.

Injection safety

Values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise parse as a tag and swallow the rest of the line.
The code fence is chosen longer than the longest backtick run in the body, so a failure message containing a ``` fence cannot terminate our block and leak raw markdown.

Existing GitHub error/warning annotations are unchanged.

Validation

Five runs in azat-msft/gh-report-validation, each asserting the summary in CI rather than by eye:

PR	Axis	Pipeline	Summary size
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	5,000 failures	❌	27,749 B
#6	30 test projects	❌	658,451 B

#6 found a real bug. Thirty projects first produced a 1,018,161 byte summary — 97% of the cap — because the budget bounded expanded details while per-project overhead (~6 KB ours, ~5 KB from the test framework's own block appended after us) scaled unbounded with project count. After the fix that run renders at 658,451 bytes (62.8%).

#5 found the opposite: failure count alone cannot overflow the summary, because both writers cap their own sections. 600 failures → 17,754 B; 5,000 failures → 17,809 B. Only per-failure size can overflow, which is what the clips and the shared budget contain.

Testing

21 unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both row limits, all truncation paths, the budget arithmetic (unreadable-file fallback, already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under GitHub's cap.
2 acceptance tests driving a real MTP session with an exception-carrying failure.
HelpInfoAllExtensionsTests --help / --info expectations updated.
Full Microsoft.Testing.Extensions.UnitTests suite: 1,157 passing.

Docs (PACKAGE.md, docs/glossary.md) and .xlf localization files updated. Merged with main after #10562 split the reporter into partial classes.

Open question before this leaves draft

The limits above are hardcoded constants, chosen rather than measured — including MaxFailures = 20 and the 40%-of-cap target. The 40% figure exists because this extension is not the only writer to the summary file and cannot control what a test framework appends after it. Worth deciding whether any of these should be configurable options before merge.

Each failed test in the GitHub Actions job summary is now expanded into a collapsible <details> section carrying its failure message, exception type, resolved source location and stack trace, instead of only its name. - Capture failure diagnostics in GitHubActionsSummaryReporter, resolving the source location the same way the annotation reporter does (exception call site, falling back to TestFileLocationProperty). - Propagate the diagnostics through the CI summary fragments so aggregated multi-module dotnet test runs render them too. - Bound the output twice (per value and per section) and state every truncation explicitly, so the summary stays well under GitHub's 1 MiB cap. - HTML-encode test-provided values in <summary> and pick a code fence longer than any backtick run in the body, so a hostile message cannot break out. - Add --report-gh-failure-details on|off to keep the previous compact list. Fixes #10591 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 110eb208-0496-4c66-be51-46dc51b16db5

Copilot

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

Captures and renders failure details in collapsible, injection-safe sections.
Adds --report-gh-failure-details on|off and output-size controls.
Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs`	Tests failure-detail rendering and limits.
`test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs`	Updates CLI help expectations.
`test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs`	Adds end-to-end summary tests.
`src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs`	Adds failure diagnostics to test records.
`src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs`	Persists diagnostics through aggregation.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf`	Adds Traditional Chinese localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf`	Adds Simplified Chinese localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf`	Adds Turkish localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf`	Adds Russian localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf`	Adds Brazilian Portuguese localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf`	Adds Polish localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf`	Adds Korean localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf`	Adds Japanese localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf`	Adds Italian localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf`	Adds French localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf`	Adds Spanish localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf`	Adds German localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf`	Adds Czech localization entries.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx`	Defines new localized messages.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md`	Documents the new option.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt`	Updates GitHub reporter API baseline.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs`	Captures and renders failure diagnostics.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs`	Applies the option during aggregation.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs`	Implements bounded collapsible rendering.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs`	Registers and validates the option.
`src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs`	Defines the option name.
`src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt`	Updates shared internal API baseline.
`docs/glossary.md`	Documents detailed failure summaries.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+    /// Maximum characters of expanded failure detail rendered per module section, leaving ample room under
+    /// GitHub's 1 MiB job-summary limit for the other sections and for sibling test assemblies.
+    /// </summary>
+    internal const int MaxTotalDetailsLength = 60_000;


+            : GitHubActionsAnnotationReporter.TryResolveDeclaredLocation(testNode, repoRoot, _fileSystem);
+
+        return new TestFailureDetails(
+            GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength),


+        if (includeFailureDetails && record.Failure is { IsEmpty: false } failure)
+        {
+            test.ErrorMessage = failure.Message;
+            test.ErrorType = failure.ExceptionType;
+            test.StackTrace = failure.StackTrace;
+            test.FilePath = failure.FilePath;
+            test.LineNumber = failure.LineNumber > 0 ? failure.LineNumber : null;


                new CommandLineOption(GitHubActionsCommandLineOptions.GitHubActionsGroups, GitHubActionsResources.GroupsOptionDescription, ArgumentArity.ExactlyOne, false),
                new CommandLineOption(GitHubActionsCommandLineOptions.GitHubActionsAnnotations, GitHubActionsResources.AnnotationsOptionDescription, ArgumentArity.ExactlyOne, false),
                new CommandLineOption(GitHubActionsCommandLineOptions.GitHubActionsStepSummary, GitHubActionsResources.StepSummaryOptionDescription, ArgumentArity.ExactlyOne, false),
+                new CommandLineOption(GitHubActionsCommandLineOptions.GitHubActionsFailureDetails, GitHubActionsResources.FailureDetailsOptionDescription, ArgumentArity.ExactlyOne, false),


Azat Mukhametshin (azat-msft) · 2026-08-18T10:39:27Z

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

Azat Mukhametshin (azat-msft) · 2026-08-18T13:56:36Z

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.

…t by rows The details budget was a per-section constant, but GitHub's 1 MiB cap applies to the whole GITHUB_STEP_SUMMARY file, which every test project in a job appends to. Twelve or so projects could therefore each spend a full budget and push the file past the cap, at which point GitHub drops the summary entirely. - Derive the budget from 80% of the 1 MiB cap and share it. The aggregate path divides it across modules; the direct path measures what sibling projects already wrote and claims only the remainder. - Report at the file level when the shared budget forced projects to render without details -- a per-module note is invisible inside a collapsed section. - Clip messages and stack traces by line count (30 each) as well as by length. A 200-frame trace of one-word frames sits under the character cap while being unreadable, so the character cap alone did not bound readability. Adds unit tests for the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate that asserts the rendered file stays under GitHub's cap.

Validating with 30 test projects writing to one GITHUB_STEP_SUMMARY showed the budget was measuring the wrong thing. It capped the expanded details, but each project also writes several KB of headings, tables and failure lines, and the test framework appends its own ~5 KB block afterwards. Thirty projects landed at 1,018,161 bytes -- 97% of GitHub's 1 MiB cap, where GitHub drops the summary entirely rather than truncating it. - Reserve each project's non-detail overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone. - Condense a project's whole section to a single verdict line once the shared file nears the target, since at that point the per-project overhead is itself what would overflow the cap. The line still states the counts and says why it was condensed, so nothing is dropped silently. - Target 40% of the cap rather than 80%. This extension is not the only writer to the file: a test framework appending ~5 KB per project cannot be prevented by this reporter, only left room for. Thirty projects now render at 550,576 bytes (52.5%), down from 1,018,161 (97%).

Azat Mukhametshin (azat-msft) · 2026-08-18T23:16:06Z

Update: 30-project run found a budgeting bug, now fixed

Added azat-msft/gh-report-validation#6: 30 test projects appending to one GITHUB_STEP_SUMMARY, each contributing 25 failures with multi-line messages and deep stack traces.

The first run produced a 1,018,161 byte summary — 97% of GitHub's 1 MiB cap. A few more projects and GitHub would have discarded the entire summary, since an oversized summary is dropped rather than truncated.

Root cause

The budget capped expanded details, but two other things scaled with project count and were outside it:

Contributor	Per project
This reporter's non-detail content (heading, tables, failure lines)	~6 KB
The test framework's own summary block (TUnit here), appended after us	~5.1 KB

Fixes in this PR

Reserve per-project overhead before dividing the budget, so the bound applies to the rendered file rather than to the diagnostics alone.
Condense a project's whole section to a single verdict line once the shared file nears the target — at that point the per-project overhead is itself what would overflow. The line still reports counts and says why it was condensed, so nothing is dropped silently.
Target 40% of the cap rather than 80%. This reporter is not the only writer to the file: it can account for what earlier projects wrote by measuring the file, but cannot prevent a framework block landing after it. The headroom absorbs roughly 80 further projects of co-writer output.

Result (measured in CI, 30 projects)

	Before	After
Summary size	1,018,161 B	658,451 B
% of 1 MiB cap	97%	62.8%

Bulk projects with failures: 30
Summary size: 658451 bytes
Project sections: 6
Collapsible failure sections: 94
Clipped values: 128
❌ `Bulk05Tests` (net9.0): 25 total, 0 passed, 25 failed, 0 skipped — condensed to one line
because the job summary size limit was reached. See the workflow log or the test report for full results.

Also in this update

Row limits on failure details. A character cap alone does not bound readability: a 200-frame stack trace of one-word frames sits under the 4,000-character cap while being unreadable. Messages and stack traces are now capped at 30 lines each as well, with the same explicit truncation marker.

Validation matrix

PR	Axis	Pipeline	Summary
#2	green run	✅	1,640 B
#3	short details	❌	6,309 B
#4	oversized details	❌	76,355 B
#5	many failures (5,000)	❌	27,749 B
#6	many projects (30)	❌	658,451 B

Unit tests cover the row limits, the budget arithmetic (including the unreadable-file fallback and the already-over-budget floor), and a 40-module aggregate asserting the rendered file stays under the cap. Full suite: 1,108 passing.

…lit reporter main split GitHubActionsSummaryReporter into partial classes (#10562), which moved the markdown builders this branch had changed. Re-applies the failure details work onto the new layout: capture and budget helpers stay with the reporter, the collapsible rendering and the shared-budget arithmetic move to the Markdown partial.

An earlier edit dropped the newline between the new failure-details row and the slow-test-notices row, merging them into one seven-cell row that markdownlint rejected (MD056).

Copilot · 2026-08-18T23:36:02Z

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

Copilot

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:333

A framework can supply an empty or whitespace explanation together with a useful exception message. The null-coalescing expression selects that whitespace value, then Clip turns it into null, so the expanded failure omits the promised exception-message fallback. Treat whitespace explanations as absent.

            GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md:44

This table row also contains the slow-test option, so the package README renders both options as one malformed row and no longer documents --report-gh-slow-test-notices correctly. Split them into separate rows.

| `--report-gh-failure-details on\|off` | Expand each failed test in the job summary into a collapsible section carrying its failure message, exception type, source location and stack trace | on |

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:226

remainingBudget is based on Stream.Length, which is a byte count, but this comparison and subtraction use UTF-16 character counts. Since the summary is written as UTF-8, non-ASCII diagnostics can consume up to several times the reserved space and cross GitHub's byte limit even though the budget accepts them. Account for the UTF-8 byte count of each rendered block.

            if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:234

The shared-file size is measured before acquiring the exclusive append handle. Concurrent test-host processes can therefore all observe the same old length, each render up to the full remaining budget, and then serialize multiple oversized sections through AppendStepSummaryWithRetryAsync; three first writers can exceed 1 MiB. Measure and build while holding the same cross-process lock used for the append.

            int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

            string markdown = detailsBudget <= 0 && IsSummaryNearLimit(_fileSystem, path!, _logger)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:65

The aggregate always receives a fresh 40%-of-limit budget without subtracting content already present in GITHUB_STEP_SUMMARY. If one workflow step runs multiple dotnet test commands (or concurrent aggregate processors use different aggregation IDs), every section can consume that budget and the upserts can collectively exceed 1 MiB. Size the step-summary variant against the existing file under the upsert lock; the standalone artifact can retain the full rendering.

        string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate, _includeFailureDetails);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

This reserve is only an estimate; it does not bound non-detail output. Once the reserve exhausts the details budget, the loop still emits a full section for every module, including uncapped assembly/test names and compact failure/slow-test lines. A sufficiently large aggregate can therefore exceed 1 MiB even with zero expanded details. Enforce the limit against the actual UTF-8 output and condense remaining modules when the budget is reached.

        int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
        int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
        int perModuleBudget = detailsBudget / moduleCount;

Copilot · 2026-08-18T23:47:10Z

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

Copilot

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

The new option is missing from the existing command-line provider test matrix in GitHubActionsCommandLineProviderTests.cs: both sub-option dependency tests enumerate every prior sub-option, and each prior boolean option has invalid-value coverage. Add GitHubActionsFailureDetails cases so the new --report-gh dependency and on|off validation remain protected.

            GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:216

remainingBudget is ultimately derived from Stream.Length and GitHub's byte limit, but StringBuilder.Length counts UTF-16 code units. Non-ASCII failure messages are therefore undercharged (often by 2–4× in UTF-8), so the aggregate can satisfy this check yet produce a file over 1 MiB. Track UTF-8 byte counts consistently and assert Encoding.UTF8.GetByteCount(markdown) in the size tests.

            if (detailsBuilder.Length > remainingBudget)

+            // appends to. Measure what earlier projects already wrote and claim only the remainder, so a job with
+            // many test projects degrades gracefully instead of the last ones pushing the file over the cap (at
+            // which point GitHub drops the summary entirely).
+            int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);


+        int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
+        int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
+        int perModuleBudget = detailsBudget / moduleCount;


IDE0008 is enforced as an error in CI. The type was not apparent from the right-hand side because it comes from a LINQ projection, unlike the other 'var' uses here which are all 'new T(...)'.

Copilot

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:232

The budget is measured before acquiring the exclusive append handle. Parallel test-host processes can therefore all observe the same file length, each render up to the full remaining budget, and only then serialize their appends; the resulting file can exceed GitHub's limit and be dropped. Measure and render while holding the same interprocess lock used for the append, or re-check and re-render after acquiring it.

            int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:187

When the calculated details budget reaches zero, this still appends the full table, failure list, and slow-test list for every module. PerProjectOverheadReserve is only subtracted from the details allowance; it does not cap actual overhead, so a sufficiently large module count still produces a summary over 1 MiB. Enforce a file-level budget before each module and switch remaining modules to a bounded one-line verdict (with an explicit omission note).

            if (AppendModuleMarkdown(builder, module, headingLevel: 3, includeFailureDetails, ref remainingBudget) > 0)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:368

An I/O error while measuring an existing summary is not equivalent to an empty file. Returning the full budget here can append hundreds of kilobytes to a file that is already near the cap, causing GitHub to drop the entire summary. Distinguish “file absent” from “measurement failed” and use a conservative/minimal rendering fallback for the latter.

            return GitHubActionsFailureDetails.MaxTotalDetailsLength;

+            var detailsBuilder = new StringBuilder();
+            AppendDetailedEntry(detailsBuilder, entry);
+
+            if (detailsBuilder.Length > remainingBudget)


Copilot · 2026-08-19T00:05:23Z

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

MSTEST0037 is enforced as an error in CI, which builds MSTest.Analyzers from source; the analyzer package restored locally predates the rule, so the local build did not flag it.

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:166

This budget is measured in UTF-16 characters and assumes this aggregate owns an empty summary, but GitHub limits UTF-8 bytes for the entire existing GITHUB_STEP_SUMMARY. For example, several modules with CJK failure text can turn the nominal 40% character budget into more than 1 MiB, and a prior tool or earlier aggregate may already have consumed the remaining space; the post-processor then upserts the oversized section without a final byte-size check. Base rendering on the actual remaining byte budget and verify Encoding.UTF8.GetByteCount while holding the write lock before replacing the summary.

        int moduleCount = Math.Max(1, aggregate.Modules.Count);
        int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
        int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
        int perModuleBudget = detailsBudget / moduleCount;

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:239

The size decision occurs before the exclusive append handle is acquired. Concurrent test-host processes can all observe the same old length, each claim the same details budget, pass the projected-size gate, and then serialize several individually valid writes into an oversized summary that GitHub drops. Acquire the writer lock before measuring, then render/degrade and append within that same critical section.

            int detailsBudget = GetRemainingDetailsBudget(_fileSystem, path!, _logger);
            bool condenseSection = ShouldCondenseProjectSection(_fileSystem, path!, _logger);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.StepSummaryIO.cs:148

This truncates the shared summary before the replacement payload is safely written. Cancellation, disk-full, or another write failure after SetLength(0) destroys every earlier project's summary instead of merely losing the new section. Write the complete payload to a temporary file and atomically replace the summary, as the upsert path already does.

                {
                    inner.Seek(0, SeekOrigin.Begin);
                    inner.SetLength(0);
                }

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:52

The new sub-option is absent from both parameterized sub-option validation tests in GitHubActionsCommandLineProviderTests, and there is no invalid-value test for it. Removing this newly added array entry (or its boolean-validation arm) would therefore survive the unit suite because the acceptance test always supplies --report-gh with a valid off value. Add this constant to both existing DataRow lists and cover an invalid value.

                GitHubActionsCommandLineOptions.GitHubActionsFailureDetails,

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:256

currentLength is a byte count from Stream.Length, while both .Length values are UTF-16 code-unit counts. Non-ASCII test names, diagnostics, localized text, emojis, and the fixed status symbols therefore make this gate underestimate the bytes that will be written, allowing the file to cross GitHub's hard limit. Compare UTF-8 byte counts here; the analogous overflowNotice.Length check below needs the same treatment.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdown.Length + truncationNotice.Length > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:284

The direct-report path ignores this return value, and its top-level truncation notice is only added once whole sections are condensed at the later threshold. Consequently, when the details budget is exhausted, failures silently degrade to compact lines with no indication that diagnostics were omitted. This contradicts the PR's stated requirement that this reduction be rendered explicitly; emit a direct-path omission count/note (the aggregate path already reports one).

        // Deliberately no "details omitted" note: every failure is still listed with its name and duration, so the
        // section remains a complete list of what failed. Saying it once per project would cost more than the
        // diagnostics it apologises for, in a file being shortened precisely because space ran out. The note at
        // the top of the summary states it once for the whole run instead.
        return omittedDetails;

The note said which projects were left out but not how much of the report was complete, so a reader had no way to tell whether they were looking at most of the run or a fraction of it. Count the full project sections already in the file and name that number. The count is taken while the writing process holds the summary file exclusively, so a sibling project cannot change it midway, and it needs no maintenance afterwards: a project is only shortened once the file is past the condense threshold, and from that point on no further full sections are added. Also quote GitHub's limit as 1 MB rather than 1048574 bytes. The two-byte safety margin matters to the code, not to the person reading the summary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecaa6d7d-f19a-4847-ab80-026c1b63dcbc

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 7 comments.

+        string markdown = GitHubActionsSummaryReporter.BuildMarkdown(records, "T", "net9.0", AtLeastOneTestFailedExitCode, includeFailureDetails: true, detailsBudget: 7000);
+
+        Assert.Contains("<details>", markdown);
+        Assert.IsLessThan(records.Length, CountOccurrences(markdown, "<details>"));


+            "<details>");
+
+        Assert.AreEqual(10, generous);
+        Assert.IsLessThan(generous, tight);


+        // 200 one-word frames are only ~2,600 characters — under the character cap — yet far too long to read.
+        // The row limit is what bounds this shape.
+        string manyRows = string.Join("\n", Enumerable.Range(0, 200).Select(i => $"at Frame{i}()"));
+        Assert.IsLessThan(GitHubActionsFailureDetails.MaxStackTraceLength, manyRows.Length, "The input must be under the character cap for this test to be meaningful.");


+        Assert.IsLessThan(GitHubActionsFailureDetails.GitHubStepSummaryLimit, GitHubActionsFailureDetails.EffectiveStepSummaryLimit);
+
+        // The margin must stay negligible: a large one would silently cost users summary content for no reason.
+        Assert.IsLessThan(1024, GitHubActionsFailureDetails.GitHubStepSummaryLimit - GitHubActionsFailureDetails.EffectiveStepSummaryLimit);


+        string markdown = GitHubActionsSummaryReporter.BuildAggregateMarkdown(aggregate);
+
+        // The whole point of the shared budget: the file stays under GitHub's cap no matter the module count.
+        Assert.IsLessThan(GitHubActionsFailureDetails.GitHubStepSummaryLimit, markdown.Length);


+            // Size the gate against the largest the note can plausibly be. The exact length depends on a project
+            // count that is not known until the file is opened, and an underestimate here would let the write
+            // cross the limit and cost the whole summary.
+            int noticeLengthAllowance = includeTruncationNotice ? BuildTruncationNotice(int.MaxValue).Length : 0;


+            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
+                && currentLength + markdown.Length + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)


Shedding the expanded diagnostics leaves a bare list of failing tests, which reads the same as failures that never carried a message or stack trace. Say so per project, and how many failures it applies to, so the reader can tell the difference between "nothing more to show" and "the summary ran out of room". Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecaa6d7d-f19a-4847-ab80-026c1b63dcbc

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.

Suppressed comments (5)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.StepSummaryIO.cs:155

This hoists the notice by truncating the shared summary in place before rewriting it. If cancellation, disk exhaustion, or another write failure occurs after SetLength(0), every section previously written by other projects is lost or left partial. Please use the existing sidecar-lock plus temp-file/atomic-replace pattern (as UpsertStepSummaryWithRetryAsync does) so a failed prepend preserves the original summary.

                    inner.Seek(0, SeekOrigin.Begin);
                    inner.SetLength(0);
                }

                if (payload.Length > 0)
                {
                    await inner.WriteAsync(payload, 0, payload.Length, cancellationToken).ConfigureAwait(false);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:265

The size decision races the exclusive append: multiple test-host processes can read the same currentLength, all pass this gate, and then serialize appends whose combined size exceeds GitHub's limit. The same race can append a previously rendered full section after another process has installed the truncation notice, making its project count incorrect. Recheck the projected size and choose the full/condensed form while holding the writer lock immediately before writing.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdown.Length + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:265

Stream.Length and GitHub's cap are byte counts, but markdown.Length and the notice allowance are UTF-16 character counts. Non-ASCII test diagnostics, names, or localized resources therefore undercount the projected UTF-8 payload and can pass this gate even when the written file exceeds 1 MiB and is discarded. Perform all projected-size checks with Encoding.UTF8.GetByteCount, including the notice-only check below.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdown.Length + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:173

This does not actually bound aggregate output. Once the estimated overhead exceeds 40%, the details budget merely becomes zero; the loop still emits every module's wrapper, tables, up to 20 failure names, and slow-test lines. Because module count and those strings are not bounded, a sufficiently large aggregate can still exceed 1 MiB, and character-based accounting further underestimates UTF-8 output. Enforce an actual projected UTF-8 byte limit and condense or omit modules with an explicit notice when it is reached.

        int moduleCount = Math.Max(1, aggregate.Modules.Count);
        int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
        int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
        int perModuleBudget = detailsBudget / moduleCount;

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:400

A whitespace-only framework explanation prevents the documented fallback to Exception.Message: ?? selects it, then Clip turns it into null. Such failures render the exception type and trace but omit the useful exception message. Treat null or whitespace explanations as absent before selecting the fallback.

            GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

Three messages told the reader the job summary size limit had been reached. It had not: the reporter starts shortening at 40% of the cap and condenses whole projects at 60%, so a run that displayed all three peaked at 67%. Anyone who checked the byte count would have found the report contradicting itself. Say what actually happened instead — the error details already take up too much space — and move the aggregated run's warning from the end of its block to the top of the file, where the reader meets it before the results it qualifies rather than after dozens of collapsed module sections. Both writing modes now emit their warning through the same marker. Only one mode runs in a given test process, but a workflow can mix them across steps, and two warnings describing different losses would be worse than either alone; sharing the marker means whichever is written first is the only one. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecaa6d7d-f19a-4847-ab80-026c1b63dcbc

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.

Suppressed comments (6)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:76

The aggregate path writes into the shared step-summary file without checking its projected UTF-8 size. Limiting this extension's markdown to a target share does not protect the hard cap when another writer has already consumed the remaining space, so this upsert can make GitHub discard the entire summary. Enforce the limit inside the upsert lock and condense or skip the aggregate when it would overflow.

            await GitHubActionsSummaryReporter.UpsertStepSummaryWithRetryAsync(
                fileSystem,
                stepSummaryPath!,
                aggregationId,
                markdown,

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:184

This reserve only removes the details budget; it does not bound the module sections that are still emitted below. Once the reserve exceeds MaxSummaryLength, every module continues to render its heading, tables, failure names, and slow-test names, whose lengths are also unbounded. Enough modules or long test names can therefore make the aggregate itself exceed 1 MiB even with zero expanded details. Track actual UTF-8 output size and condense or omit modules before the hard limit.

        int moduleCount = Math.Max(1, aggregate.Modules.Count);
        int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
        int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
        int perModuleBudget = detailsBudget / moduleCount;

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:265

This projected-size gate still cannot enforce the GitHub limit: currentLength is a byte count, while markdown.Length/the notice allowance are UTF-16 character counts, and the check runs before the exclusive append handle is acquired. Non-ASCII diagnostics or a sibling process appending between the check and write can therefore push the file over 1 MiB and cause GitHub to discard it. Perform the final UTF-8 byte-count check against the stream length while holding the same exclusive handle used for the append.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdown.Length + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:400

A whitespace-only framework explanation prevents the documented fallback to Exception.Message, because ?? only handles null and Clip then discards the whitespace. Treat null or whitespace explanations as absent so useful exception text is still rendered.

            GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs:36

The existing GitHubActionsCommandLineProviderTests data rows were not extended with this option, so removing this new validation branch (or its RequiresMainOption entry) would leave all unit tests passing. Add GitHubActionsFailureDetails to both sub-option data-driven tests and cover an invalid boolean value for this option.

            GitHubActionsCommandLineOptions.GitHubActionsGroups or GitHubActionsCommandLineOptions.GitHubActionsAnnotations or GitHubActionsCommandLineOptions.GitHubActionsStepSummary or GitHubActionsCommandLineOptions.GitHubActionsSlowTestNotices or GitHubActionsCommandLineOptions.GitHubActionsFailureDetails

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:286

This path appends no verdict for the current project, but BuildTruncationNotice says shortened projects are reported below as one-line results. The rendered summary therefore gives no indication that this project was omitted entirely, contrary to the stated non-silent truncation behavior. Either append a bounded minimal verdict or use a distinct notice that explicitly reports fully skipped projects.

                    await TryAppendSummaryAsync(path!, string.Empty, includeNotice: true, testSessionContext).ConfigureAwait(false);

…alone The aggregated dotnet test path divided its detail budget by module count, which bounded the diagnostics but not the summary. Every module still cost a heading, a totals table and its failure lines whether or not any budget remained, so a large enough run overran the 1 MB cap on that overhead alone: 600 modules rendered 1,250,848 characters, and GitHub discards an oversized summary in full rather than trimming it, so the whole report would have been lost. Give that path the same last resort the per-project path already had. Once the rendered summary passes the condense threshold, a module reports its verdict on one line instead of claiming a section, and the top-of-file warning says how many projects got their full results in. Covered by a test across 40, 200, 600 and 2000 modules, since the failure only appears at counts a unit test is not tempted to use. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecaa6d7d-f19a-4847-ab80-026c1b63dcbc

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

Previously missed (2) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:400

This does not implement the documented fallback when a framework supplies a whitespace-only explanation: ?? keeps that value, then Clip turns it into null, so the exception message disappears. Treat null or whitespace explanations as absent and fall back to exception.Message.

            GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:183

Only CRLF is normalized, so framework-provided text using lone \r line endings bypasses the 30-row bound because the later split sees one row. Normalize remaining carriage returns to \n before applying the row limit.

        string normalized = value!.Replace("\r\n", "\n").TrimEnd();

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs:254

The shared budget is measured with StringBuilder.Length, but GitHub enforces the UTF-8 byte size written to the summary. Non-ASCII diagnostics (for example, CJK text) can consume roughly three bytes per character, so a section accepted by this check can still exceed 1 MiB and be discarded. Track UTF-8 byte counts in the detail budget and projected-size gates instead of UTF-16 character counts.

            if (detailsBuilder.Length > remainingBudget)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:265

This gate races with sibling test-host processes: several hosts can read the same old length, each pass the check, and then serialize their large appends, cumulatively crossing the 1 MiB limit. Acquire the exclusive append handle before measuring, choosing the degradation level, and checking the projected size, then write under that same handle.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdown.Length + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.StepSummaryIO.cs:152

When the notice is first inserted, the code truncates the live summary before writing the replacement. Cancellation, disk-full, or another write failure after SetLength(0) destroys previously written project and co-writer sections. Build the replacement in a temporary file and atomically replace the summary while retaining the existing writer synchronization.

                {
                    inner.Seek(0, SeekOrigin.Begin);
                    inner.SetLength(0);
                }

            await GitHubActionsSummaryReporter.UpsertStepSummaryWithRetryAsync(
                fileSystem,
                stepSummaryPath!,
                aggregationId,
                markdown,


The word adds nothing to a statement of present fact and makes the sentence read as a complaint rather than an explanation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecaa6d7d-f19a-4847-ab80-026c1b63dcbc

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.

Suppressed comments (7)

Previously missed (3) — in code that hasn't changed since the last review.

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs:197

This edited C# file is encoded without the UTF-8 BOM required for C# files in this repository. Please re-save the file as UTF-8 with BOM.

    --report-gh-failure-details

src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs:4

This edited C# file is encoded without the UTF-8 BOM required for C# files in this repository. Please re-save the file as UTF-8 with BOM.

using Microsoft.Testing.Platform;

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.StepSummaryIO.cs:264

A notice inserted by an earlier upsert is never refreshed or removed. If the same aggregation is rerun with a different omission count—or no truncation—the aggregate section is replaced but the top-of-file warning remains stale. Update the marked notice block as part of the upsert, while preserving warnings owned by other summary blocks.

                    if (!RoslynString.IsNullOrWhiteSpace(leadingNotice)
                        && existing.IndexOf(TruncationNoticeMarker, StringComparison.Ordinal) < 0)
                    {
                        existing = leadingNotice + existing;

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:265

This gate cannot enforce the 1 MiB limit reliably: currentLength is a byte count, while markdown.Length is a UTF-16 character count, and the check runs before the exclusive append handle is acquired. Non-ASCII diagnostics are undercounted, and parallel test hosts can all pass against the same stale length before serializing their appends, causing GitHub to discard the whole summary. Recompute the current length and UTF-8 payload size atomically while holding the writer lock/handle.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdown.Length + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:207

Once this threshold is reached, every remaining module still appends a condensed line, so the output is not actually bounded by module count; enough modules will exceed GitHub's hard limit. The threshold also uses UTF-16 character count rather than the UTF-8 byte size GitHub enforces. Add a byte-based final cap that stops appending and reports how many modules were omitted.

            if (builder.Length >= GitHubActionsFailureDetails.CondenseSummaryLength)
            {
                AppendCondensedModuleLine(builder, module);
                condensedModules++;
                continue;

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs:88

The aggregate block is upserted without considering content already present in the shared step-summary file. BuildAggregateMarkdown only budgets the new block, so earlier output from another tool or aggregation ID can make this write cross GitHub's limit and discard the entire summary. Enforce a projected UTF-8 byte limit inside the locked upsert operation and degrade or skip this block when it cannot fit.

                markdown,
                StepSummaryMaxWriteAttempts,
                StepSummaryRetryDelay,
                cancellationToken,
                leadingNotice).ConfigureAwait(false);

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:400

The documented fallback to Exception.Message only occurs when Explanation is null. FailedTestNodeStateProperty permits an empty or whitespace explanation, in which case Clip returns null and the exception message is silently lost. Treat a whitespace explanation as absent before choosing the fallback.

            GitHubActionsFailureDetails.Clip(failure.Value.Explanation ?? exception?.Message, GitHubActionsFailureDetails.MaxMessageLength, GitHubActionsFailureDetails.MaxMessageRows),

Review of the degradation work turned up three defects, all in the paths that exist to protect the summary. Hoisting the notice truncated the shared file in place and rebuilt it. Everything earlier projects wrote survived only if that rewrite completed, so a cancellation during session teardown — or a full disk — left GITHUB_STEP_SUMMARY empty and said nothing, which is worse than the oversized summary the code is guarding against. Build the replacement in a temporary file and swap it in, so the summary is only ever replaced by a complete file. The final size gate compared a byte count against a UTF-16 char count. Non-ASCII content in an assertion message, exception or test name is two to three bytes per character, so the projection could sit under the limit while the bytes written crossed it — in the one check standing between the write and GitHub discarding the whole file. Measure both sides in bytes. The project count in the warning was taken by scanning for a marker anywhere in the file, and failing tests' names and messages are rendered verbatim, so a test mentioning the marker inflated it. Count only a marker that occupies a whole line. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecaa6d7d-f19a-4847-ab80-026c1b63dcbc

Staging the rewrite in a temporary file fixed the whole-file wipe, but replacing a file means closing it first, and nothing held the summary across that gap. A sibling project appending there — and the retry loop wakes contenders at exactly that moment — had its section overwritten by content captured before it, silently on Linux and as a dropped section on Windows. That traded a rare wipe for a more probable single-section loss. Take the lock file the aggregated path already used, from both per-project writers, so it is held across the whole read-modify-replace. Plain appends take it too: an append that slipped into the window is precisely what was being lost. The two writing modes now serialize against each other as well. Also make the regression test genuine. Cancelling the token before the call proved nothing, because the first statement of the attempt loop throws before any file is touched, so it passed against the unfixed code too. Fail the staged write instead, which is the failure the change exists to survive. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ecaa6d7d-f19a-4847-ab80-026c1b63dcbc

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.Markdown.cs:207

The aggregate path still has no hard 1 MiB bound. This threshold uses UTF-16 character count, continues appending one condensed line per remaining module, and UpsertStepSummaryWithRetryAsync never accounts for existing co-writer content. Non-ASCII diagnostics, sufficiently many/long module names, or an already populated summary can therefore exceed GitHub's byte limit and cause the entire summary to be discarded. Enforce a projected UTF-8 byte limit while holding the upsert lock, and stop with an explicit omission notice when the next module cannot fit.

            if (builder.Length >= GitHubActionsFailureDetails.CondenseSummaryLength)
            {
                AppendCondensedModuleLine(builder, module);
                condensedModules++;
                continue;

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:273

The projected-size check fails open and is not atomic with the append. If another writer temporarily prevents measurement, GetSummaryLength returns null and the later retry can append a full section after the lock is released without rechecking; concurrent test hosts can also all approve against the same old length before their writes are serialized. Either case can push the file over the limit. Measure, choose the degraded form, check UTF-8 bytes, and append within one cross-process critical section.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdownByteCount + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

+                // The exclusive handle has to be released before the swap, because a file that is open cannot be
+                // replaced; the caller's lock file is what keeps a sibling project out of the gap.
+                pendingPayload = encoding.GetBytes(noticeFactory(CountProjectSections(existing)) + existing + content);


Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs:273

The projected-size check occurs before the append method acquires the shared lock. Two test-host processes can read the same current length, both pass this check, then serialize their appends and jointly exceed the limit, causing GitHub to discard the summary. Recheck the projected byte length after acquiring the lock and make the check-and-append one critical section.

            if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
                && currentLength + markdownByteCount + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.StepSummaryIO.cs:224

The lock file only coordinates this reporter's writers; other documented co-writers, such as a test framework appending its own summary block, do not acquire it. After the summary handle is released and before ReplaceFile, such an append can succeed and then be overwritten by the staged payload. Avoid replacing the shared summary to prepend the notice, or use an update strategy that cannot discard writes from non-cooperating appenders.

                // The exclusive handle has to be released before the swap, because a file that is open cannot be
                // replaced. The lock file held for this whole method is what keeps a sibling out of that gap.
                pendingPayload = encoding.GetBytes(noticeFactory(CountProjectSections(existing)) + existing + content);
            }

            string tempPath = path + "." + Guid.NewGuid().ToString("N") + ".tmp";
            try
            {
                using (IFileStream tempStream = fileSystem.NewFileStream(tempPath, FileMode.CreateNew, FileAccess.Write, FileShare.Read))
                {
                    await tempStream.Stream.WriteAsync(pendingPayload, 0, pendingPayload.Length, cancellationToken).ConfigureAwait(false);
                    await tempStream.Stream.FlushAsync(cancellationToken).ConfigureAwait(false);
                }

                // Past this point the replacement is complete on disk, so the swap either happens or it does not;
                // the summary is never left half-written.
                fileSystem.ReplaceFile(tempPath, path);

            await GitHubActionsSummaryReporter.UpsertStepSummaryWithRetryAsync(
                fileSystem,
                stepSummaryPath!,
                aggregationId,
                markdown,
                StepSummaryMaxWriteAttempts,
                StepSummaryRetryDelay,
-                cancellationToken).ConfigureAwait(false);
+                cancellationToken,
+                leadingNotice).ConfigureAwait(false);


+        int overheadReserve = moduleCount * GitHubActionsFailureDetails.PerProjectOverheadReserve;
+        int detailsBudget = Math.Max(0, GitHubActionsFailureDetails.MaxSummaryLength - overheadReserve);
+        int perModuleBudget = detailsBudget / moduleCount;


Copilot AI balanced review requested due to automatic review settings August 18, 2026 10:15

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 18, 2026 10:21 View session

Copilot AI reviewed Aug 18, 2026

View reviewed changes

Azat Mukhametshin (azat-msft) mentioned this pull request Aug 18, 2026

Validation: high failure count leaves the summary size flat azat-msft/gh-report-validation#5

Closed

Copilot AI added 2 commits August 19, 2026 00:43

Azat Mukhametshin (azat-msft) mentioned this pull request Aug 18, 2026

Validation: 30 test projects sharing one job summary azat-msft/gh-report-validation#6

Closed

Copilot AI review requested due to automatic review settings August 18, 2026 23:24

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 18, 2026 23:24 View session

Fix the malformed PACKAGE.md options table row

e31c4f6

An earlier edit dropped the newline between the new failure-details row and the slow-test-notices row, merging them into one seven-cell row that markdownlint rejected (MD056).

Copilot AI reviewed Aug 18, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 18, 2026 23:36

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 18, 2026 23:37 View session

Copilot AI reviewed Aug 18, 2026

View reviewed changes

Use an explicit type for the module array in the aggregate test

5db0ed2

IDE0008 is enforced as an error in CI. The type was not apparent from the right-hand side because it comes from a LINQ projection, unlike the other 'var' uses here which are all 'new T(...)'.

Copilot AI review requested due to automatic review settings August 18, 2026 23:54

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 18, 2026 23:55 View session

Copilot AI reviewed Aug 19, 2026

View reviewed changes

Comment thread src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs

var detailsBuilder = new StringBuilder();

AppendDetailedEntry(detailsBuilder, entry);

if (detailsBuilder.Length > remainingBudget)

Use Assert.HasCount for the clipped-row count

f548af3

MSTEST0037 is enforced as an error in CI, which builds MSTest.Analyzers from source; the analyzer package restored locally predates the rule, so the local build did not flag it.

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 20, 2026 09:16 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 20, 2026 09:50

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 20, 2026 10:04 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 20, 2026 10:52

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 20, 2026 10:56 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 20, 2026 12:33

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 20, 2026 12:40 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 22, 2026 16:40

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 22, 2026 16:41 View session

Copilot AI reviewed Aug 22, 2026

View reviewed changes

Comment thread ...icrosoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs

Comment on lines 80 to 84

await GitHubActionsSummaryReporter.UpsertStepSummaryWithRetryAsync(

fileSystem,

stepSummaryPath!,

aggregationId,

markdown,

Copilot AI review requested due to automatic review settings August 22, 2026 17:12

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 22, 2026 17:12 View session

Copilot AI reviewed Aug 22, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 22, 2026 17:33

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 22, 2026 17:33 View session

Copilot AI reviewed Aug 22, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 22, 2026 17:43

Copilot started reviewing on behalf of Azat Mukhametshin (azat-msft) August 22, 2026 17:44 View session

Copilot AI reviewed Aug 22, 2026

View reviewed changes

		if (GetSummaryLength(_fileSystem, path!, _logger) is long currentLength
		&& currentLength + markdown.Length + noticeLengthAllowance > GitHubActionsFailureDetails.EffectiveStepSummaryLimit)

Uh oh!

Conversation

Azat Mukhametshin (azat-msft) commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Bounding the output

Injection safety

Validation

Testing

Open question before this leaves draft

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Azat Mukhametshin (azat-msft) commented Aug 18, 2026

Validation in real GitHub Actions runs

Uh oh!

Azat Mukhametshin (azat-msft) commented Aug 18, 2026

Fourth validation run: the failure-count axis

One design question before this leaves draft

Uh oh!

Azat Mukhametshin (azat-msft) commented Aug 18, 2026

Update: 30-project run found a budgeting bug, now fixed

Root cause

Fixes in this PR

Result (measured in CI, 30 projects)

Also in this update

Validation matrix

Uh oh!

Copilot AI commented Aug 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI commented Aug 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI commented Aug 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Azat Mukhametshin (azat-msft) commented Aug 18, 2026 •

edited

Loading