feat(reporter): and printOnlyFailures and --reporter-only-failures - #42766
Devin Rousso (dcrousso) wants to merge 2 commits into
Conversation
passing tests add unnecessary reporter output allow `dot` and `line` to suppress progress while retaining failure details and the final summary
4d99d63 to
2c834d2
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
| configDir: string, | ||
| _mode?: 'list' | 'test' | 'merge', | ||
| _commandHash?: string, | ||
| _onlyFailures?: boolean, |
There was a problem hiding this comment.
we can make this public for others to be able to respect the mode
| const entry = this.fileDurations.get(fileAndProject) || { duration: 0, workers: new Set() }; | ||
| entry.duration += result.duration; | ||
| entry.workers.add(result.workerIndex); | ||
| this.fileDurations.set(fileAndProject, entry); |
There was a problem hiding this comment.
I think we still want the durations, etc, there probably is a better place to bail later on.
| export class BlobReporter extends TeleReporterEmitter { | ||
| private readonly _messages: JsonEvent[] = []; | ||
| private readonly _attachments: { originalPath: string, zipEntryPath: string }[] = []; | ||
| private _messages: JsonEvent[] = []; |
There was a problem hiding this comment.
I think blob report should always get all the data, blob is not a reporter, it is a database that is used to replay and generate actual reports off.
| }, | ||
| }; | ||
| for (const test of this.suite.allTests()) | ||
| for (const test of this.suite.allTests().filter(test => !this._onlyFailures || isFailure(test))) |
There was a problem hiding this comment.
this filtration is duped and is tedious, let's unify it across the reporters. i'm thinking filters and filtrating visitors to reuse and keep existing code as much as possible.
| this._resultIndex.set(result, index); | ||
|
|
||
| if (!this.screen.isTTY) | ||
| if (this._printOnlyFailures || !this.screen.isTTY) |
There was a problem hiding this comment.
List will be hard to implement - we need to what, delay the report of begin?
reporter options must be configured individually add `--reporter-only-failures` to filter terminal output and final reports - preserve blob data and complete run accounting - expose `onlyFailures` for custom reporters - add `printOnlyFailures` support to `list`
2c834d2 to
6c46ad8
Compare
Test results for "tests 1"2 failed 8 flaky52423 passed, 1250 skipped Merge workflow run. |
🟢 CI is clear — both failures are pre-existing macOS CDN infra noiseHi, I'm the Playwright bot and I took a look at the failing CI. The only two failures are the DetailsPre-existing flake / infra
The other 8 entries in the report are flaky (rescued on retry) across video, cookie, emulation, goto, leaks and ui-mode specs — unrelated to reporters, and not failures. Triaged by the Playwright bot - agent run |
Test results for "MCP"1 failed 8628 passed, 1446 skipped Merge workflow run. |
🟢 CI is clear — the one failure is a known Firefox-on-Windows flakeHi, I'm the Playwright bot and I took a look at the failing CI.
DetailsPre-existing flake / infra
Why the diff can't reach it This PR is scoped to Triaged by the Playwright bot - agent run |
2 participants
passing tests add unnecessary reporter output
allow configuring
dotandlineto suppress progress while retaining failure details and the final summaryalso add a generic
--reporter-only-failuresthat does the same but across all reporters and only for that specific runfixes #42702