Conversation
|
Claude finished @nicobytes's task in 3m 0s —— View job PR Review — strict-gate CI wiring
I focused on the code that landed after the two bot reviews — chiefly the new first-parent base resolution in New IssuesNone blocking.
Everything else in the workflow logic checks out:
Resolved
The implementation is careful and the test suite is genuinely strong (equality on an extracted line, boundary values pinned, malformed-input guard tested). The one open item above is already tracked as a follow-up; the merge-queue attribution caveat (an earlier-queued PR's violation can eject the group) is a documented, accepted design tradeoff, not a defect. · branch |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings remain in CI wiring, reporting, and pre-push behavior.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Activates the diff-scoped strict TypeScript gate for pull requests and adds local pre-push enforcement.
Changes:
- Wires the gate into Maven CI for pull requests.
- Adds pre-push enforcement.
- Adds duration reporting, tests, documentation, and specifications.
File summaries
| File | Description | Review findings |
|---|---|---|
specs/37536-wire-strict-gate-ci/spec.md |
Requirements and acceptance criteria | No final review comments. |
specs/37536-wire-strict-gate-ci/data-model.md |
Outcome and duration model | Nit (2 votes): Documents report.files.length, which the implementation does not expose and cannot use for the documented diff-size calculation. |
specs/37536-wire-strict-gate-ci/contracts/ci-hooks.md |
CI and hook contracts | No final review comments. |
core-web/tools/scripts/strict-gate/README.md |
Gate documentation and status | Nit (1 vote): Contradictory “spike” and “wired and live” status wording. |
core-web/tools/scripts/strict-gate/lib/format.mjs |
Duration and summary formatting | Moderate (1 vote): Omits unmapped files from both summary branches. Moderate (3 votes): Reports assignment counts rather than unique changed files, excluding unmapped files from the measurement. |
core-web/tools/scripts/strict-gate/format.duration.test.mjs |
Duration summary tests | No final review comments. |
core-web/pom.xml |
Pull-request Maven gate execution | Critical (1 vote): Hard-codes origin/main, producing incorrect or failed comparisons for master pull requests.Moderate (1 vote): GitHub output lacks the scope/dependency-discard preamble. |
core-web/.husky/pre-push |
Local strict-gate enforcement | Moderate (3 votes): Runs the toolchain without a changed-path preflight for non-frontend pushes. Moderate (1 vote): Mislabels exit-2 execution failures as violations. Moderate (1 vote): Relies on potentially stale origin/main, diverging from CI. |
Review details
Suppressed comments (6)
core-web/.husky/pre-push:52
- The hook ignores the refs supplied on pre-push stdin and always checks
HEAD. A validgit push origin other-local-branch:remote-branch(or a multi-ref push) can therefore be allowed or refused based on a different commit than the one being sent. Parse the local ref/SHA from stdin, skip deletions, and pass that SHA as--head, or explicitly restrict the hook to current-branch pushes.
pnpm exec node tools/scripts/strict-gate/run.mjs \
--base origin/main \
--head HEAD \
core-web/.husky/pre-push:68
- For exit
2(for example, an unresolved base or unreadable project graph), the harness may print no violations, but this wrapper still tells the developer to “Fix the violations above.” That misclassifies a harness failure and obscures the recovery path; distinguish findings (1) from a gate execution error before printing this guidance.
echo "Push refused by the strict-gate check (exit $status)."
echo " Fix the violations above, or push anyway with: git push --no-verify"
core-web/.husky/pre-push:51
- This local invocation assumes
origin/mainis current, but the hook never refreshes the remote-tracking ref; CI explicitly fetchesorigin/mainimmediately before Maven. A developer with a stale ref can therefore get a different result from the pull-request run, violating the documented claim that local and remote answers agree. Refresh the target base (or make the freshness prerequisite explicit and enforce it) before running the check.
pnpm exec node tools/scripts/strict-gate/run.mjs \
--base origin/main \
core-web/pom.xml:518
--format=githubemits only::errorannotations;formatGithub()does not emit the required scope/dependency-discard preamble. The new CI log therefore starts with findings without telling agents that only changed lines are in scope (the separate Markdown summary does not change the formatter output). Add the scope preamble to the GitHub formatter before enabling this invocation.
<argument>--format=github</argument>
core-web/tools/scripts/strict-gate/README.md:80
- This new section says the gate is "Wired and live", but the unchanged introduction still says "This is spike output, not production tooling" (line 9). After adding CI and pre-push enforcement, that description is contradictory and can lead contributors to treat the live hooks as unsupported or removable spike artifacts. Update the introduction/status wording to reflect that this is now temporary production tooling.
**Wired and live since #37536.** Two invocation points, deliberately asymmetric:
core-web/tools/scripts/strict-gate/lib/format.mjs:146
formatMarkdownis the job summary used by the new--format=githubCI path, but the pass branch never rendersreport.unmapped(and the findings branch omits it too). A changed TypeScript file that no project claims therefore appears as a clean pass with no indication that it was unexamined, contrary to the gate's edge-case contract; include the unmapped files/reasons in both summary branches.
costLine(report)
- Files reviewed: 8/8 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| <argument>exec</argument> | ||
| <argument>node</argument> | ||
| <argument>tools/scripts/strict-gate/run.mjs</argument> | ||
| <argument>--base=${git.origin.branch}</argument> |
| pnpm exec node tools/scripts/strict-gate/run.mjs \ | ||
| --base origin/main \ | ||
| --head HEAD \ | ||
| --flags strict \ | ||
| --granularity line \ | ||
| --scope core-web \ | ||
| --format text |
| const files = report.targets.reduce((n, t) => n + t.files.length, 0); | ||
| return `_Checked ${files} file(s) across ${report.targets.length} project config(s) in ${seconds}s._`; |
| | Elapsed time | `report.durationMs.total`, already computed by the harness | Never printed today — FR-025 surfaces it | | ||
| | Diff size | `report.files.length` | Pairs the cost with what produced it; a duration without it is not comparable across pull requests | |
e4cebb7 to
c7ada9a
Compare
The strict-gate harness merged with #37403 was wired into nothing and has never executed once. Since 2026-09-08 no pull request has been annotated, reported on, or blocked by it, and main has kept accumulating non-strict TypeScript exactly as before. This wires it into CI as a blocking gate: a change that adds a strict-mode violation on a line it wrote does not merge. A `strict-gate` execution in core-web/pom.xml, inside the existing `validate` profile beside lint-test and format-test. `successCodes` lists 0 and nothing else -- findings (1) and a harness that could not run (2) both fail. A 180s timeout fails the same way, bounding the harness's unshallow git-fetch fallback. It runs in two contexts, and both are load-bearing for different reasons. `pull_request` is where the author reads it: the harness emits `::error file=,line=,col=` lines that GitHub renders inline on the changed lines, plus a job summary. `merge_group` is where it is ENFORCED. This repository declares no required status checks on main -- the ruleset requires a pull request, one approval, thread resolution and signed commits -- so a red check on a pull request does not by itself stop a merge. A job that fails in the merge queue ejects the pull request, and that does. Wiring only the first would have produced a gate that goes red and merges anyway. Trunk and nightly stay skipped: HEAD equals origin/main there, so the diff is empty. Maven property activation has no OR, hence two profiles rather than one condition. No workflow file changes; activation reads the runner's own GITHUB_EVENT_NAME. `useMavenLogger` is pinned false with a comment even though it is the default, because that default is the only reason the `::error` lines reach the log at column 0 where GitHub can render them. Setting it true kills every annotation while leaving the build green. One output-only harness change: every run states its cost in the job summary. The gate now sits on the critical path of every frontend merge, so the real runtime distribution matters; the harness already measured every run and simply never printed it. What it decides is unchanged. Known gap, accepted knowingly: there is no escape hatch. A frontend change that legitimately must add a violation cannot merge until it is fixed or the gate is switched off repository-wide. The spike measured 0 false positives across its corpus, so the expected frequency is low, and inventing a bypass before anyone needs one tends to produce the bypass everybody uses. Scope: CI only. No local git hook; core-web/.husky/ and lint-staged.config.mjs are untouched, so no contributor's local workflow changes. No new dependency. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
c7ada9a to
973da59
Compare
…typecheck-gate-into-ci-merged-on-main-but-never-executes
…hook reference (#37536) Addresses review on #37545. The cost line counted (config -> file) assignments, not changed files. `selectConfigs` claims a source under EVERY eligible config, so a project whose lib and spec configs both include a file produced two target entries for one changed file, and `targets[].files.length` summed them -- reporting a one-file diff as two. Unmapped files were dropped from the count entirely. Both defeat the only reason that line exists: to be the cost-versus-diff-size evidence for the runtime question. Now counts distinct paths across targets and unmapped. The original test never caught this because it used distinct files across targets. Two new cases pin the two failure modes directly. The job summary also never mentioned unmapped files, so a changed TypeScript file that no project compiles read as a clean pass with nothing said about it -- the edge case the spec names, and a worse one now the gate blocks. The summary now lists them with the reason, in both the passing and the failing branch. Not a failure signal: the difference between "nothing was wrong" and "nothing was looked at". Also removed a comment in core-web/pom.xml pointing at core-web/.husky/pre-push, a file this pull request does not add. It was left behind when the local hook was cut, and would have sent a developer looking for something that was never there. Reconciled two related staleness bugs: the harness README called itself "not production tooling" next to a section saying it is wired and live, and data-model.md documented `report.files.length` as the diff-size source when buildReport exposes no such field. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks — triaged all of it. Fixed in db56f5b. Important context for anyone reading the bot reviews above: both were produced against earlier commits. Copilot reviewed the first push, which still contained Fixed🟠 Dangling 🟡 You were also right that the existing test never exercised it — it used distinct files across targets. Two new cases pin the two failure modes directly, and both were confirmed failing before the fix. 🟡 🟡 🟡 README contradiction — correct. "This is spike output, not production tooling" sat directly above a section saying it is wired and live. Reconciled. Not changed, with reasons🔴/🟡 Two things make me want to leave it: the property is the pre-existing 🟡 That is a real concern and it is exactly what the harness's own design notes worry about. I am leaving it because changing the shape of the annotation output is more than the output-only carve-out this PR is operating under, and it is a harness design question from #37403 rather than something this wiring introduced. Worth its own issue if it bites. Verification
|
| <argument>exec</argument> | ||
| <argument>node</argument> | ||
| <argument>tools/scripts/strict-gate/run.mjs</argument> | ||
| <argument>--base=${git.origin.branch}</argument> |
There was a problem hiding this comment.
The gate ran in this PR's own CI (job 104166954128) and printed:
strict-gate: warning — no merge base between 'origin/main' and head;
comparing against the tip of 'origin/main' instead.
Findings may include changes the base introduced.
The cause looks structural rather than incidental: cicd_comp_test-phase.yml:259 checks out at fetch-depth: 1, and the compensating step at :266 is git fetch --depth=1 origin main:refs/remotes/origin/main. Both sides are depth-1, so git merge-base can't traverse, and the harness takes its documented fallback in lib/changed-files.mjs:81-94.
Is that fallback acceptable now that the gate blocks? The harness's own comment above that code says it is the path that "produced the run that reported 50 findings, essentially none of them the branch's own". It was survivable while the only other --base=origin/main consumer was nx affected, because over-including projects for lint is harmless on a lint-clean main. But main is deliberately not strict-clean — that's the premise of #37198 (workspace-wide strict migration) — so every line main moved ahead of the PR's merge ref gets strict-checked and can fail the build with violations the author cannot fix.
On merge_group this looks worse rather than better: HEAD there is the queue's temp commit sitting on main plus every PR ahead of this one, so with no merge base those other PRs' .ts changes appear as this PR's diff. Would a strict violation in an earlier-queued PR eject this one, annotated on lines its author never wrote?
Worth noting the "0 false positives across the corpus" figure was measured with a correct merge base, so I don't think it transfers to the configuration the gate actually ships in.
One option that keeps the shallow checkout — derive the base from the event payload:
env:
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
run: |
git fetch --depth=1 origin main:refs/remotes/origin/main
if [ -n "$BASE_SHA" ]; then git fetch --depth=1 origin "$BASE_SHA"; fiplus -Dgit.origin.branch=$BASE_SHA on that suite's Maven args. With the base already at the true divergence point the base-tip fallback becomes exactly correct rather than a degradation — and it makes the hardcoded origin/main-vs-master point raised separately moot. Alternatively fetch-depth: 0 with filter: blob:none gives full history without the ~1.1 GB pack the comment below warns about.
There was a problem hiding this comment.
Confirmed and fixed in 61f3fde. This was the one that would have made the gate unusable, and your diagnosis of the cause was exactly right — I reproduced the warning in the job log before changing anything.
Took the event-payload option. cicd_comp_test-phase.yml now resolves pull_request.base.sha || merge_group.base_sha, fetches that single commit shallow, and hands it to the gate through a new strict.gate.base property. With the base already at the divergence point the base-tip fallback stops being a degradation and becomes exactly right — no deepening, no extra pack.
Two deliberate choices inside that:
strict.gate.baseis its own property, not a change to${git.origin.branch}. Your framing is why: lint and format tolerate an approximate base, this does not. Repointing the shared property would have silently changed both neighbours, and widening the blast radius is not something this change should do quietly. It defaults to${git.origin.branch}, so a local run is unchanged.- The fetch failing is not a new failure mode. If the base commit cannot be fetched the step emits a
::warningand leaves the gate onorigin/main— today's behaviour, not a hard fail.
You are also right that this settles the hardcoded-origin/main point you raised separately: a pull request targeting something other than main now compares against its own base. I have closed that thread pointing here.
On merge_group: merge_group.base_sha is the commit the queue group is based on, so the diff still spans every pull request in the group. A violation introduced by an earlier-queued pull request can therefore eject the group. I think that is defensible — the queue is testing "main plus these changes" and that combination genuinely has a violation, and the annotation names the file and line — but it is not clean attribution and I would rather it be written down than discovered. Noted in the workflow comment.
The point about "0 false positives" not transferring is the one I would most want on the issue, so I have added it there.
There was a problem hiding this comment.
Follow-up on the master half of this, which your other comment raised separately: 61f3fde makes it moot rather than fixing it directly. The base now comes from pull_request.base.sha, so the base branch's name stops mattering — a pull request targeting anything compares against its own base.
For the record, origin/master does not currently exist (git ls-remote --heads origin master is empty), so the case was latent rather than active. It is now structurally unreachable instead of merely unused.
| the finding is wrong, that is a harness bug worth an issue. | ||
| --> | ||
| <successCodes> | ||
| <successCode>0</successCode> |
There was a problem hiding this comment.
successCodes listing only 0 is the change I'd most want recorded on the issue before this merges.
Issue #37536 says verbatim under Scope decision: "Non-blocking (reporting only) in this issue. The blocking flip stays out of scope", its acceptance list carries the exact XML with <successCode>0</successCode><successCode>1</successCode>, and "flipping to blocking" is named under Out of scope. The spike (#37401 / PR #37403) recommended shipping non-blocking first for the runtime reason — 8.4-9.4s average, 12s tail against a 10s budget — and the README in this PR repeats those numbers unchanged.
The PR description argues the reversal and I find the argument persuasive on its merits. My concern is procedural: the issue is the reviewer-facing contract and it currently says the opposite of what ships, with no visible re-approval, and the "observed runtime from at least five real pull requests" that the issue wanted before a blocking decision doesn't exist yet.
Would you update #37536's scope and acceptance list to match, so the decision is recorded where the next reader looks, rather than only in this PR's description?
There was a problem hiding this comment.
Fair, and the procedural objection is the right one to raise — the issue is the reviewer-facing contract and it currently contradicts what ships.
For the record of how it moved: the issue scoped this non-blocking, I built it that way, and the reversal came from the author asking for a gate that actually prevents the merge. That is a legitimate change of mind, but you are right that it lives only in this PR's description, with no re-approval and without the five-pull-request runtime evidence the issue wanted first.
I will update #37536's Scope decision and acceptance list to match what ships, and record the merge-base finding above in the same edit — that one materially changes the picture, because the spike's precision number was measured under a different configuration than the gate was about to ship in.
Holding the merge until that is done and someone has looked at it.
| 0 = clean, and that is the only outcome that passes. | ||
|
|
||
| 1 = new strict-mode violations on lines this pull request | ||
| wrote. 2 = the harness could not run at all. Both fail, and |
There was a problem hiding this comment.
This comment states the doctrine clearly: "a gate that reports 'clean' without having looked is worse than no gate, because it is indistinguishable from a clean pull request."
ensureBaseRef applies that rigorously to an unresolvable base — it throws rather than report an empty diff. But an unusable base (resolvable, yet with no common ancestor) only prints a warning to stderr and proceeds. Combined with useMavenLogger=false, that warning lands as an unprefixed line in the middle of a six-minute Maven log.
Isn't that the same class of silent degradation this comment rules out? Would it be better for run.mjs to exit 2 when the base resolution falls back to base-tip — optionally behind --require-merge-base, default on for --format=github — and emit it as ::error:: so it surfaces?
One caveat if you take this: it's only safe to land together with a fix for the base-SHA problem, otherwise it red-lights every frontend PR on day one.
There was a problem hiding this comment.
Good distinction, and I think you are right that the two cases are not being treated consistently: an unresolvable base throws, an unusable one warns and proceeds. Under useMavenLogger=false that warning is an unprefixed line in the middle of a long log, which is close to not reporting it at all.
I have not made the change, and your own caveat is why — landing it before the base fix would red-light every frontend pull request on day one. Now that 61f3fde supplies a real base, the fallback should be rare enough that failing on it is safe rather than catastrophic.
I would rather not do it in this pull request though. Exiting 2 on base-tip changes the harness's decision logic, which is the one line this change has held to not crossing (the elapsed-time line is output-only and was called out as the single exception). It also wants its own verification: a case where the base resolves but has no common ancestor, which none of the current tests construct.
Proposing it as a follow-up issue rather than folding it in — with --require-merge-base defaulting on for --format=github, as you suggest. Tell me if you would rather it block this one.
| ...report.targets.flatMap((t) => t.files), | ||
| ...report.unmapped.map((u) => u.path) | ||
| ]).size; | ||
| return `_Checked ${changed} changed file(s) across ${report.targets.length} project config(s) in ${seconds}s._`; |
There was a problem hiding this comment.
The verb here reads as contradicting the note printed two lines below it. With one mapped and one unmapped file the summary renders:
_Checked 2 changed file(s) across 1 project config(s) in 12.4s._
**1 changed file(s) were not examined** — no project configuration claims them:
Two adjacent lines asserting the opposite thing about the same file. Beyond the wording, this number is the SC-005 evidence the duration is meant to be normalised against, so a count that includes files the compiler never touched isn't the cost-per-checked-file it reads as.
The JSDoc just above shows the intent was understood ("they were changed, they just were not examined") — it looks like only the verb didn't follow. Would something like this work?
return `_${changed} changed file(s) across ${report.targets.length} project config(s), ${seconds}s._`;or, if the split is worth surfacing: _Checked ${changed - report.unmapped.length} of ${changed} changed file(s) …_
Note this would mean updating the assertion in format.duration.test.mjs too.
There was a problem hiding this comment.
Agreed — two adjacent lines asserting the opposite thing about the same file. Fixed in 78980a7 by dropping the verb entirely:
_2 changed file(s) across 1 project config(s), 4.0s._
Went with your first option rather than Checked X of Y. The note below already states how many were not examined, so the split would have said it twice, and Checked 3 of 3 on the common case reads like a puzzle. The line now states diff size and duration; what was skipped is the note's job.
Test assertions updated to exact equality on that string, per your other comment.
| if (report.unmapped.length === 0) return null; | ||
| const rows = report.unmapped.map((u) => `- \`${u.path}\` — ${u.reason}`).join('\n'); | ||
| return [ | ||
| `**${report.unmapped.length} changed file(s) were not examined** — no project configuration claims them:`, |
There was a problem hiding this comment.
Two questions about this note now that the gate blocks.
exitCode is derived solely from findings (lib/report.mjs:60), and unmapped is read nowhere else in the harness — so a changed .ts that no project config claims produces a green check and merges. The spec asks for these to be reported rather than failed, so the implementation is compliant and I'm not calling it a bug. But an unmapped file is precisely a file the gate did not look at, and it's the one such case resolved in favour of passing — which sits awkwardly next to the "worse than no gate" principle quoted in the POM.
Second, formatGithub is unchanged (correctly, since a note has no file/line to attach to), so this produces no annotation at all. It lives only in the job summary, which is the page nobody opens on a green run. So the case most in need of being seen is in the place least likely to be looked at.
Would a ::warning annotation for unmapped changed sources be worth adding, so it surfaces on a passing run? And is exit 0 still the right call now that this is the enforcement point rather than a report?
There was a problem hiding this comment.
Both questions are fair and I have only half-answered them.
Fixed: the note now renders in the findings branch too, not just the passing one. Your other comment caught that the two unmapped tests both used the clean fixture, so the findings branch was unenforced — there is a case for it now.
Not fixed, deliberately: exit code and annotations. Adding a ::warning means touching formatGithub, and changing exit 0 means touching report.mjs — both are the harness's decision logic, which this change has held to not crossing. The output-only carve-out was explicitly scoped to the elapsed-time line.
But I think you are right on the merits, and the asymmetry you describe is real: the one case where the gate demonstrably did not look is the one resolved in favour of passing, and it reports only to the page nobody opens on a green run. That sits badly next to the "worse than no gate" line I quoted in the POM.
Rolling it into the same follow-up as the base-tip question, since both are "the harness is quiet about something it should not be quiet about". A ::warning seems clearly right; whether it should fail is the harder call and I would rather it be taken with the unmapped rate measured rather than guessed.
|
|
||
| // 3 files across 2 project configs. A duration without the diff size it came from is not | ||
| // comparable across pull requests, which makes it useless for the measurement SC-005 wants. | ||
| assert.match(summary, /\b3\b[^|\n]*file/i, 'the summary must state how many files were checked'); |
There was a problem hiding this comment.
I reconstructed both sides of this change in a sandbox and ran six mutations of the implementation against this suite. Four survive:
- project-config count replaced with
new Set(targets.map(t => t.project)).size— 7/7 pass - duration emitted on a separate line from the file count, defeating this test's stated purpose ("paired with the size of the diff that produced it") — 7/7 pass
- the blank line before the findings table removed, so GitHub stops rendering it as a table — 7/7 pass
unmappedNotedropped from the findings branch entirely — 7/7 pass
That last one is because both unmapped tests use the clean fixture, so "a changed file nobody compiled must be named" is enforced on pass and unenforced on fail — the branch an author is actually reading.
The common cause looks like the whole-summary regexes: formatMarkdown returns a multi-line document, and unmappedNote emits **N changed file(s) were not examined**, the same number-then-file shape these regexes look for. I confirmed a false pass — with targets: [] and two unmapped files, a costLine that drops unmapped renders Checked 0 changed file(s) and /\b2\b[^|\n]*file/i still matches, satisfied by the note.
Would exact equality on an extracted cost line be preferable? It's also how the sibling suite asserts (hunks.test.mjs, report.contract.test.mjs use assert.deepEqual/assert.equal, reserving assert.match for free-text fields deliberately left unpinned):
const costLineOf = (summary) => {
const line = summary.split('\n').find((l) => l.startsWith('_Checked '));
assert.ok(line, 'the summary must carry a cost line');
return line;
};
assert.equal(costLineOf(summary), '_Checked 3 changed file(s) across 2 project config(s) in 9.0s._');That single change closes this and the two findings below it. A findings-branch case carrying both a finding and an unmapped entry would close the other two mutations.
Separately: the PR checklist says "4 new unit tests" — the file contains 7 (# tests 7 from the runner). Worth correcting so a reviewer counting against the claim doesn't go looking for a missing file.
For what it's worth, the TDD claim checks out: I ran this file against the pre-PR lib/format.mjs and got 0 pass / 7 fail.
There was a problem hiding this comment.
This is the most useful review comment I have had on anything. Rewritten in 78980a7.
You are right about the cause: whole-document regexes against a multi-line document, where the unmapped note renders the same number-then-file shape the assertions were looking for. I reproduced your targets: [] + two-unmapped case and confirmed the false pass — the assertion was satisfied by the note while the cost line read 0 changed file(s).
Took the costLineOf suggestion, and it does close all four mutations. The suite is now equality against one extracted line, which as you note is what hunks.test.mjs and report.contract.test.mjs already do. I also added the two cases you identified as missing: a findings-branch fixture carrying both a finding and an unmapped entry, and an explicit assertion that a blank line precedes the findings table.
11 tests now. Corrected the "4 new unit tests" claim in the PR description — thank you for counting.
| const summary = formatMarkdown(report); | ||
|
|
||
| assert.match(summary, /\b1\b[^|\n]*file/i, 'one changed file, claimed twice, is still one file'); | ||
| assert.doesNotMatch(summary, /\b2\s*file/i); |
There was a problem hiding this comment.
I think this assertion can't fail for the bug it was written to catch.
The bug it targets is summing targets[].files.length. I ran that mutation — the buggy output is:
_Checked 2 changed file(s) across 2 project config(s) in 5.0s._
/\b2\s*file/i requires 2, optional whitespace, then literally file — but the real string has changed in between, so the regex doesn't match and doesNotMatch passes against the buggy output. Node's failure report confirmed only line 121 fired; this line never did.
It's also the only assertion in the file with no message string, which may be how it escaped notice.
Delete it as redundant with line 121, or pin the actual phrasing?
assert.doesNotMatch(summary, /Checked\s+2\s+changed file/i, 'the two config entries must not be counted as two files');There was a problem hiding this comment.
You are right, and it is worse than redundant — it could never have failed for the bug it was written to catch. /\b2\s*file/i needs 2, optional whitespace, then file; the real output is 2 changed file, so \s* never spans changed. I confirmed the same way you did: only the line above ever fired.
Deleted rather than re-pinned. The assertion above it is now equality against the extracted cost line, which covers this case exactly and without a regex that can drift out of sync with the wording again.
Your observation about it being the only assertion with no message string is a good tell — I will treat a bare assertion as a smell in this file from now on.
| test('a sub-second run is not reported as 0s', () => { | ||
| // The no-op case — a pull request touching no frontend file — costs ~0.3s. Rounding that to | ||
| // "0s" would make the cheapest and most common case invisible in the evidence. | ||
| const report = reportWith({ durationMs: { total: 331, typescript: 0, templateAware: 0 } }); |
There was a problem hiding this comment.
This fixture may be picking a value that was never at risk. (0.331).toFixed(1) is "0.3" — three rounding steps clear of the boundary. The value that actually produces the forbidden output is anything under 50ms: I confirmed total: 40 renders in 0.0s.
The premise in the comment also looks off by roughly 2x. I timed the real no-op path (node run.mjs --base HEAD --head HEAD, which early-returns after resolveChangedFiles) three times: durationMs.total came back 173.2 / 176.8 / 179.9 ms, so ~0.2s rather than ~0.3s. That's within a factor of four of the 0.0s cliff, which makes it reachable on a faster runner with a warm git cache.
So "sub-second runs must not round to zero" is asserted for a safe value while the implementation genuinely does render 0.0s at the fast end. Would you test the boundary instead, and either accept 0.0s and soften the comment, or switch costLine to sub-second precision below a threshold?
One more on line 162: doesNotMatch(summary, /\b0\.0\s*s\b/) scans the whole document, so it forbids 0.0s anywhere. I mutated costLine to append a legitimate (typescript 0.0s) breakdown and this test failed for a reason unrelated to rounding.
There was a problem hiding this comment.
Confirmed, and the implementation was genuinely wrong here — not just the fixture. I wrote your boundary case as a test first and it failed: total: 40 rendered 0.0s.
So rather than move the fixture, I changed costLine. Sub-second runs now render in milliseconds:
_0 changed file(s) across 0 project config(s), 40ms._
Your timing is the reason — ~175ms measured on the real no-op path is a factor of four from the cliff, which is reachable, and the no-op is the most common outcome the evidence has. Rounding the commonest case to zero would have quietly hollowed out the measurement.
Also took the scoping point on line 162: the assertion now runs against the extracted cost line rather than the whole document, so a legitimate 0.0s elsewhere in the summary cannot trip it. And added a companion case pinning that a run at exactly 1000ms still renders as 1.0s, so the two branches of the threshold are both held.
| stop a merge. A job that fails in the merge queue ejects the pull request, and that | ||
| is the only place a failing gate actually blocks. | ||
|
|
||
| Trunk and nightly runs stay skipped: HEAD equals origin/main there, so the diff is |
There was a problem hiding this comment.
Is this the actual mechanism? The skip is GITHUB_EVENT_NAME not matching pull_request or merge_group — the diff is never computed on trunk or nightly, because the execution doesn't run at all.
The distinction matters because the stated reason is false for cases that are nonetheless skipped. Both cicd_3-trunk.yml and cicd_4-nightly.yml also accept workflow_dispatch, and a dispatch from a feature branch has HEAD != origin/main. cicd_4-nightly.yml:139 sets change-detection: 'disabled', so the frontend filter defaults to true and the Frontend Unit Tests job runs with -Pvalidate on every nightly — it's the event name, not an empty diff, keeping the gate quiet there.
As written, this invites a future reader to enable push on the belief it would be a no-op. Would something like this be more accurate?
"Every other event leaves skip.strict.gate at true. On trunk/nightly that is also what you'd want — HEAD is origin/main, so there would be nothing to report — but the skip is by event name, not by diff emptiness."
The same sentence appears in the README's "Where this runs" section.
There was a problem hiding this comment.
You are right, and the wording invited exactly the mistake you describe. Corrected in 78980a7.
I checked each of your claims before rewriting: cicd_3-trunk.yml and cicd_4-nightly.yml do both accept workflow_dispatch, and cicd_4-nightly.yml:139 does set change-detection: disabled, so the frontend suite runs nightly with -Pvalidate regardless of paths. The skip there is the event name and nothing else.
The comment now says so, and carries the warning forward explicitly — that adding a push profile on the belief it would be a harmless no-op is wrong for cases that are nonetheless skipped today. I folded in the cicd_5-lts.yml point from your other comment in the same paragraph, since it is the same trap.
The README sentence you flagged has been reconciled too.
| /** | ||
| * What the run cost, paired with the diff size that produced it. | ||
| * | ||
| * Emitted on every run, passing ones included. The gate blocks, so its cost sits on the critical |
There was a problem hiding this comment.
"Emitted on every run" may be broader than what happens. costLine is only called from formatMarkdown, never from formatGithub — and --format=github is what the POM passes, so formatGithub is what reaches stdout and the build log. The cost line arrives only via run.mjs's separate GITHUB_STEP_SUMMARY write, which format.mjs doesn't itself check.
This PR's own CI run bears that out: the gate printed nothing to the log except the merge-base warning.
So for anyone reading annotations or the Maven log, the cost line never appears. Would you narrow the wording, e.g.:
"Emitted on every markdown-formatted run, passing ones included. Note that formatGithub — the format this repo's CI invocation uses — does not call this; the cost line reaches GITHUB_STEP_SUMMARY via run.mjs's separate markdown write, not stdout or annotations."
There was a problem hiding this comment.
Correct, and the wording was overclaiming. Narrowed in 78980a7 to say where it does not appear — that formatGithub, which is the format this repo's CI invocation passes, never calls this, so the line reaches the run page only through run.mjs's separate GITHUB_STEP_SUMMARY write, never stdout and never an annotation.
Your point that this PR's own run bears it out is the part that made me fix the comment rather than argue with it: the log carried nothing but the merge-base warning.
That merge-base warning is now fixed in 61f3fde, and separately I have added a note to the follow-up issue about the gate being silent on a clean run — you raised the same thing on the pom.xml:509 thread and it is the more useful half of this observation.
| * and most common outcome invisible in the evidence. | ||
| */ | ||
| function costLine(report) { | ||
| const seconds = (report.durationMs.total / 1000).toFixed(1); |
There was a problem hiding this comment.
Two smaller things about this line.
report.durationMs.total has no guard. It isn't reachable through run.mjs today — buildReport defaults durationMs and both collectDiagnostics return paths populate all three fields — but formatMarkdown is exported, and buildReport's parameter default only fires on undefined, not on a partial object. A durationMs: {} renders in NaNs. in the job summary. Given this file's own stance that a plausible-looking wrong number is the dangerous failure mode, would a guard be worth the line?
const ms = report.durationMs?.total;
const seconds = Number.isFinite(ms) ? (ms / 1000).toFixed(1) : '?';Separately: neither costLine nor unmappedNote reaches formatText, which is the CLI default and therefore what a developer gets from the local invocation the README recommends. FR-018 only mandates the summary so this is in spec, but the reasoning in the new JSDoc ("the real distribution is worth knowing") seems to apply equally to the local run. Deliberate scope call, or worth the two lines?
There was a problem hiding this comment.
Both taken, in 78980a7.
The NaN guard is in, phrased as you suggested — and you are right that this file's own stance is what makes it worth the line: a summary reading in NaNs. is precisely the plausible-looking wrong number it exists to avoid. I added a test pinning it, which renders ?s rather than NaNs.
On formatText: leaving it out for now, and I want to be straight that this is a scope call rather than a principled one. You are right that the reasoning applies equally to the local run — a developer running the README's recommended invocation gets no cost line at all. It is two lines. I am holding it only because formatText is untouched by this change today and I would rather not grow the diff while the merge-base fix is the thing under review. Say the word and I will add it.
| Both are needed, for different reasons. The pull-request run is where inline | ||
| annotations render on the diff, so it is what an author actually reads. The | ||
| merge-queue run is what ENFORCES: this repository declares no required status checks | ||
| on main (the ruleset requires a pull request, one approval, thread resolution and |
There was a problem hiding this comment.
I checked this claim against gh api repos/dotCMS/core/rulesets and it is accurate right now: the active ruleset on main carries required_signatures, pull_request (1 approving review, thread resolution required) and non_fast_forward, and defines no required_status_checks.
Two things worth knowing, since this premise is what the whole two-profile design rests on:
- The ruleset providing it is an org-level one named
2026-08-24_incident-response, created 2026-08-24. The name suggests a reactive posture rather than the durable state of branch protection. - A separate ruleset,
Default Merge Queue, does definerequired_status_checks(semgrep, Finalize/Final Status, Initialize/Initialize) but currently hasenforcement: disabled.
So the claim holds today, but only while that incident-response ruleset stays active and the checks-based one stays disabled. Would it be worth noting that dependency here, so whoever re-enables Default Merge Queue sees that this design assumed the opposite?
There was a problem hiding this comment.
Thank you for actually checking this against the API rather than taking the comment's word — the whole two-profile design rests on this premise and I had verified it only as a snapshot.
Both caveats are now in the comment: that the ruleset providing it is the org-level 2026-08-24_incident-response, and that Default Merge Queue does define required_status_checks with enforcement: disabled. The point about the name suggesting a reactive posture rather than durable state is the one worth preserving, so the comment says the premise holds today and names what would invalidate it.
Whoever re-enables that ruleset now has a chance of seeing that this design assumed the opposite.
| it "[INFO] ", and silently kills every annotation while the | ||
| build stays green and the job summary still looks correct. | ||
| --> | ||
| <useMavenLogger>false</useMavenLogger> |
There was a problem hiding this comment.
The PR description is careful to say a green build shouldn't be read as proof the gate ran — and it did run, which I confirmed in the log. But I don't think this run exercises the mechanism this comment is protecting.
This PR changes zero .ts files, so the harness returned in 0.37s having compiled nothing. The ::error file=... at column 0 to inline-annotation chain, the exit-1 blocking path, and the new cost line are all still unexercised, and the issue's acceptance criterion "a pull request touching core-web TypeScript shows inline annotations on the diff and a job summary" therefore has no evidence behind it yet.
Before merging something that can eject PRs from the merge queue, would you run one throwaway branch carrying a deliberate TS7006 and confirm three things: exit 1, the annotation actually rendering on the diff, and the behaviour under merge_group? A screenshot on the issue would close the AC.
Related observability point: on the clean run the gate printed nothing but the merge-base warning — there's no way to distinguish "ran and was clean" from "ran and had nothing to look at" from the log alone. A one-line success summary on stdout would make that visible the moment it matters.
There was a problem hiding this comment.
You are right that the run proves less than the description implies, and the merge-base finding on your other thread has since made that gap matter more rather than less.
To be precise about what the run did establish: the strict-gate execution appears in the log at line 1448, so the two-profile activation works and the gate is no longer dead code, which was the open question when I opened this. Everything past that point — annotations rendering on a diff, exit 1 ejecting from the queue, the cost line — is unexercised, exactly as you say, because this branch changes no .ts.
I will run the throwaway branch with a deliberate TS7006 and post the three confirmations plus a screenshot on #37536 before this merges. Doing it after 61f3fde rather than before, so it exercises the base the gate will actually ship with.
Locally I have confirmed exit 1 fails the build under GITHUB_EVENT_NAME=merge_group (BUILD FAILURE, Process exited with an error: 1) and that the annotation reaches the Maven log unprefixed at column 0 — but that is my machine, not the thing you are asking for, and the inline rendering step is precisely the part a local run cannot demonstrate.
Your observability point is the one I had not considered: on a clean run the gate prints nothing, so "ran and was clean" and "ran and had nothing to look at" are indistinguishable from the log. That is a small version of the failure this whole issue is about. Added to the follow-up.
| <plugin> | ||
| <groupId>org.codehaus.mojo</groupId> | ||
| <artifactId>exec-maven-plugin</artifactId> | ||
| <configuration> |
There was a problem hiding this comment.
Is this block needed? The profile's plugin entry merges into the base <build> plugin block, so the base plugin-level configuration at lines 123-166 already applies to strict-gate.
I verified with ./mvnw help:effective-pom -pl :dotcms-core-web -Pvalidate under GITHUB_EVENT_NAME=merge_group: the effective strict-gate execution shows both the PATH and NODE_OPTIONS=--max-old-space-size=6144 from the base, even though this block never mentions NODE_OPTIONS. Deleting these seven lines appears to change nothing. (The pre-existing executions are unaffected either way — all six still show the identical executable, workingDirectory and NODE_OPTIONS.)
The reason I'd rather see it go than stay: it duplicates PATH in two places without duplicating the comment at lines 126-131 explaining why the Node dir is prepended. A future edit to the base PATH (a second dir, a different shim) silently wouldn't reach this copy, and nothing would fail — the strict-gate would just pick up the runner's default Node.
If you'd rather keep it belt-and-braces, would a one-line comment noting the plugin-level config is inherited from the base <build> be worth adding?
There was a problem hiding this comment.
Verified and removed in 78980a7. I reproduced your help:effective-pom check under GITHUB_EVENT_NAME=merge_group and the effective strict-gate execution does carry NODE_OPTIONS even though the block never mentioned it, which proves the base configuration merges in and the block was doing nothing.
Your reason for preferring it gone is the one that convinced me: it forked the PATH comment at 126-131 into a copy that a future edit to the base would silently miss, and the failure would be invisible — the gate would just pick up the runner's default Node.
Replaced with a comment saying the plugin-level configuration is inherited and why there is deliberately none here, so the next person does not re-add it.
Worth noting a mistake I made fixing this: my first pass replaced the identical block in the pre-existing format profile as well. Caught it before pushing and restored it byte-for-byte; core-web/pom.xml is now purely additive against main.
| </goals> | ||
| <phase>generate-resources</phase> | ||
| <configuration> | ||
| <skip>${skip.strict.gate}</skip> |
There was a problem hiding this comment.
lint-test and format-test key off ${skip.validate}; this keys off a new property. So -Pvalidate -Dskip.validate=true — the documented way to turn off the other two checks — disables lint and prettier but leaves the blocking strict gate running.
I assume that's intentional, given the design goal that the gate be hard to opt out of. But it isn't mentioned in the property comment at lines 26-38, and the issue's acceptance criterion asked specifically for <skip>${skip.validate}</skip>, so this is a deviation that isn't among the two the PR description declares.
Also worth stating explicitly somewhere: a plain local ./mvnw -Pvalidate runs lint and prettier but not this, unlike its two neighbours. The comment at 463-474 covers the intent; it's the asymmetry with skip.validate specifically that I think needs a sentence. Something like: "-Dskip.validate=true does not turn this off; it has its own property by design."
There was a problem hiding this comment.
Intentional, undocumented, and you are right that it is an undeclared deviation — the issue's acceptance list does ask for <skip>${skip.validate}</skip> verbatim.
The intent is what you assume: a blocking gate that the documented way to silence its two neighbours also silences is not much of a gate. But that reasoning was in my head and nowhere else.
Added to the property comment in 78980a7, close to your phrasing, and with the flip side stated too — that a plain local ./mvnw -Pvalidate runs lint and prettier but not this, unlike its neighbours. I will add it to the PR description's deviations list as a third entry rather than leaving it implied.
| only a pathological case can, chiefly the harness's | ||
| unshallow git-fetch fallback when the base ref cannot be | ||
| resolved (a full fetch here is ~1.1GB / ~20min). Timing out | ||
| kills the process with 143, which is not a success code, so |
There was a problem hiding this comment.
I pulled META-INF/maven/plugin.xml from exec-maven-plugin 3.0.0 to check this, and the parameter and the 180s bound are both real. Two notes on the mechanism the comment describes.
The build does fail on timeout, but not through successCodes — the watchdog path throws MojoExecutionException from the ExecuteException branch ("Timeout. Process runs longer that N ms.") and never consults successCodes at all. Harmless as written, but it teaches a mechanism that isn't there: someone adding <successCode>143</successCode> to "allow timeouts" would find it has no effect.
Second, and more of a question than an assertion: commons-exec's watchdog destroys the direct child (pnpm), not the node grandchild that pnpm exec spawns. If the grandchild survives holding the stdout pipe, PumpStreamHandler.stop() can block joining its pump thread — which would mean the 180s ceiling isn't the hard wall described here. I didn't exercise this, and I'm flagging it only because boundedness is the whole justification for the parameter. If you want certainty cheaply, keeping the Maven timeout as a backstop and adding an internal deadline inside run.mjs would do it.
There was a problem hiding this comment.
Corrected in 78980a7 — you are right that the comment taught a mechanism that is not there, and the concrete consequence you name (someone adding <successCode>143</successCode> to allow timeouts and finding it does nothing) is exactly the kind of thing a wrong comment causes. It now says the watchdog throws directly and never consults successCodes.
The grandchild question I have not resolved, and I would rather say so than hand-wave it. You are right that commons-exec destroys pnpm, not the node process pnpm exec spawns, and that a surviving grandchild holding the stdout pipe could block PumpStreamHandler.stop(). I have not exercised it either.
It matters for the reason you give: boundedness is the entire justification for the parameter, and I have been describing it as a hard wall. Until it is tested it is a soft one.
Adding an internal deadline inside run.mjs would settle it, but that is harness decision logic again. Cheaper interim option, if you think it is worth it: invoke node directly rather than through pnpm exec, so there is no grandchild — the PATH already has the installed Node prepended, which is what the pnpm exec indirection was buying. Happy to do that here if you agree it is equivalent; otherwise it goes in the follow-up with the deadline.
|
|
||
| Activation is on the runner's own event name, so no workflow file needs to change: | ||
| cicd_1-pr.yml triggers on `pull_request` and cicd_2-merge-queue.yml on `merge_group`, | ||
| and the runner exports GITHUB_EVENT_NAME for each. Trunk and nightly export something |
There was a problem hiding this comment.
Worth one more sentence here about the branches this doesn't cover.
cicd_5-lts.yml triggers on push: branches: [release-*] and calls the test phase with the frontend filter (line 92), so the Frontend Unit Tests job does run with -Pvalidate on LTS branches — the gate is skipped there only because the event is push.
That's the right outcome (diffing an LTS branch against origin/main would be meaningless, and git.origin.branch is hardcoded to origin/main), but it looks accidental rather than designed. Would you note it, so nobody later "fixes the coverage gap" by adding a push profile and quietly starts diffing release branches against main?
For completeness: PRs targeting LTS branches never reach this workflow at all, since cicd_1-pr.yml:26-29 restricts to branches: [main, master].
There was a problem hiding this comment.
Added in 78980a7, folded into the same paragraph as your trunk/nightly point since it is the same trap.
I verified the trigger: cicd_5-lts.yml does run the frontend suite with -Pvalidate on release-* pushes, and the gate is quiet there only because the event is push.
You are right that the outcome is correct but looks accidental. The comment now names LTS explicitly in the list of things that would break if someone "fixed the coverage gap" by adding a push profile — diffing a release branch against origin/main being the specific nonsense that would result.
The note that PRs targeting LTS branches never reach this workflow is useful context I did not have; thank you.
| `across ${report.targets.length} project config(s).`, | ||
| '', | ||
| costLine(report), | ||
| ...(unmappedNote(report) ? ['', unmappedNote(report)] : []) |
There was a problem hiding this comment.
Small readability point: unmappedNote(report) is invoked twice on each branch — once as the predicate, once as the value — and the same shape repeats at line 183. Would hoisting it once above the if (report.exitCode === 0) read better?
const note = unmappedNote(report);
// ...
...(note ? ['', note] : [])It's pure and cheap so there's no correctness or performance consequence.
Separately, on line 166 just above: the pre-existing sentence already ends with "across N project config(s)", and costLine now repeats that verbatim two lines later, so the pass summary reads:
No new strict-mode violations. 0 pre-existing or dependency diagnostic(s) ignored, across 1 project config(s).
_Checked 1 changed file(s) across 1 project config(s) in 12.4s._
Would dropping the config count from the first sentence and letting costLine own it be cleaner? The cost line carries the file count too, so it's the more informative of the two.
There was a problem hiding this comment.
Both taken in 78980a7.
unmappedNote is hoisted to a single const note above the branch, exactly as you wrote it. You are right there is no correctness consequence — it just reads as though the call might be expensive or non-deterministic, which it is not.
The duplicated "across N project config(s)" is a better catch than it looks. I dropped it from the pre-existing sentence and let the cost line own it, on your reasoning: the cost line carries the file count too, so it is the more informative of the two. The pass summary now reads:
No new strict-mode violations. 0 pre-existing or dependency diagnostic(s) ignored.
_1 changed file(s) across 1 project config(s), 12.4s._
That is a change to a line that predates this PR, so flagging it rather than burying it.
…omments (#37536) Second round of review on #37545. Nothing here changes the gate's decision logic; it is all output correctness, test strength, and comments that taught mechanisms that do not exist. The test suite was weak in a way worth describing, because the reviewer found it by mutation testing rather than by reading. Four separate mutations of the implementation passed the whole suite: replacing the config count with a distinct-project count, splitting the duration onto its own line (defeating the stated purpose of pairing it with the diff size), removing the blank line that makes the findings table render, and dropping the unmapped note from the findings branch entirely. The common cause was whole-document regexes: formatMarkdown returns a multi-line document, and the unmapped note renders the same number-then-"file" shape the assertions searched for, so an assertion could be satisfied by a line other than the one under test. One assertion -- `doesNotMatch(/\b2\s*file/i)` -- could never fail for the bug it targeted, because the real output reads "2 changed file" and `\s*` does not match " changed ". Assertions are now equality against an extracted cost line, which is how the sibling suites already work. Two real bugs came out of that. The sub-second test asserted a value three rounding steps clear of the boundary; `toFixed(1)` renders anything under 50ms as "0.0s", and the measured no-op path is ~175ms -- a factor of four, reachable on a faster runner. Sub-second runs now render in milliseconds. And `durationMs.total` was unguarded: formatMarkdown is exported and buildReport's default only fires on undefined, so a partial object rendered "NaNs" in the job summary. The cost line also read "Checked N changed file(s)" directly above a note saying some of them were not examined -- two adjacent lines asserting opposite things about the same file. The verb is gone; the count is what it always was. Three POM comments taught mechanisms that are not there: - the skip was described as trunk/nightly having an empty diff. It is by event name. Trunk and nightly also accept workflow_dispatch from a feature branch, nightly disables change detection entirely, and cicd_5-lts.yml runs the frontend suite on release-* pushes -- so "it would be a harmless no-op" is false for cases that are nonetheless skipped. - the timeout was described as failing through successCodes. The watchdog throws directly and never consults them, so adding a success code to "allow timeouts" would do nothing. - the "no required status checks" premise holds today, but the ruleset providing it is an org-level `2026-08-24_incident-response`, and a separate `Default Merge Queue` ruleset does define required checks with enforcement disabled. Whoever re-enables it should see that this design assumed the opposite. Also removed the plugin-level <configuration> duplicated into the validate profile: verified against `help:effective-pom` that the base <build> entry already supplies executable, workingDirectory, PATH and NODE_OPTIONS -- the copy omitted NODE_OPTIONS and forked the PATH comment, so a future edit to the base would have silently missed it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This is the finding that would have made the gate unusable, caught in review on #37545 and reproduced in this PR's own CI run (job 104166954128): strict-gate: warning — no merge base between 'origin/main' and head; comparing against the tip of 'origin/main' instead. The cause is structural, not incidental. cicd_comp_test-phase.yml checks out at fetch-depth: 1 and compensates with `git fetch --depth=1`, so both sides of the comparison are truncated, `git merge-base` cannot traverse, and the harness takes its documented base-tip fallback. That fallback is survivable for `nx affected` -- over-including projects for lint is harmless on a lint-clean main -- which is why it went unnoticed while that was the only consumer. It is not survivable for a BLOCKING strict check, because main is deliberately not strict-clean while #37198 is in flight: every commit main moved ahead of the branch gets attributed to the pull request and strict-checked, so authors would be rejected for violations they never wrote. The harness's own comment on that code path records it producing 50 findings, essentially none of them the branch's own. The spike's "0 false positives" figure was measured with a correct merge base and does not transfer. Fix: resolve the base from the event payload (pull_request.base.sha, or merge_group.base_sha), fetch that one commit shallow, and hand it to the gate through a new `strict.gate.base` property. With the base already AT the divergence point, the base-tip fallback stops being a degradation and becomes exactly right -- no deepening, no extra fetch cost. `strict.gate.base` is deliberately its own property rather than a change to ${git.origin.branch}: lint-test and format-test tolerate an approximate base, this gate does not, and widening the blast radius to the two neighbouring checks is not something this change should do quietly. It defaults to ${git.origin.branch}, so a local run behaves exactly as before. Side effect worth naming: this also settles the hardcoded-origin/main concern raised separately in review. A pull request targeting a branch other than main now compares against its own base rather than against main. If the base commit cannot be fetched the step warns and leaves the gate on origin/main, which is today's behaviour rather than a new failure mode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Both of the things I owed you are moving. Probe: #37564 — draft, DO NOT MERGE. Carries this branch plus one deliberate Scope recorded on #37536 — #37536 (comment). It states plainly that the blocking flip was a direct decision during implementation and was not re-approved against the issue's stated scope, lists all four deviations from the acceptance list, and carries the merge-base finding including the part I think matters most for anyone reading the original evidence: the "0 false positives" figure was measured with a correct merge base and does not transfer to the configuration the gate was about to ship in. That framing is yours. Deliberately not in this pull request, tracked for a follow-up, each answered on its own thread: exiting Thank you for the mutation testing — four of my assertions were passing for the wrong reason and one could never have failed at all. That is not something I would have found by rereading. |
|
Probe evidence is in: #37536 (comment) (see the following comment). All three confirmed — base from the event payload, annotation anchored to #37564 closed and its branch deleted. One thing the probe turned up that I would not have predicted: the base-tip warning still fires, and now fires on every run, because both sides remain shallow so From my side this is ready. Holding for your call on whether any of the three follow-ups should block here. |
…7536) The gate was handed github.event.pull_request.base.sha as its base. GitHub pins that field when the pull request is opened and never moves it as the base branch advances, so on this branch it was 276 commits stale. CI checks out refs/pull/N/merge at depth 1, merge-base cannot traverse, and the gate fell back to diffing the stale base against that merge commit -- scoping in 190 files, 97 of them .ts/.html belonging to other people's merged work, and reporting ~50 pre-existing strict violations against this author. Use the checkout's own first parent instead. The merge commit CI builds has the current base tip as parent 1 and the branch as parent 2, so parent1..HEAD is exactly the pull request's contribution. Deepening the fetch by one makes that parent visible; requiring a second parent keeps it honest, since on a non-merge checkout HEAD^1 would be the previous commit rather than the base. Verified against the merge commit that failed (3568f08): the first parent is 6fe9551 and the scoped diff is precisely this pull request's 8 files, none of them .ts/.html, so the gate reports PASS. Re-running it with the stale base reproduces the CI findings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…typecheck-gate-into-ci-merged-on-main-but-never-executes
Proposed Changes
Fixes #37536. Supersedes #37448.
PR #37403 merged the strict-gate harness to
mainon 2026-09-08 and wired it into nothing. Since then no pull request has been annotated, reported on, or blocked by it, andmainhas kept accumulating non-strict TypeScript exactly as before.This wires it into CI as a blocking gate: a change that adds a strict-mode violation on a line it wrote does not merge.
core-web/pom.xmlstrict-gateexecution in thevalidateprofile + two activation profilescore-web/tools/scripts/strict-gate/lib/format.mjscore-web/tools/scripts/strict-gate/format.duration.test.mjscore-web/tools/scripts/strict-gate/README.mdCI only. No local git hook —
core-web/.husky/andlint-staged.config.mjsare untouched, so no contributor's local workflow changes. No workflow file changes. No new dependency.It runs in two contexts, and both are load-bearing
pull_request::error file=,line=,col=lines render inline on the changed lines, plus a job summary.merge_groupThat second row is the part worth reviewing. This repository declares no required status checks on
main— the ruleset requires a pull request, one approval, thread resolution and signed commits, and no checks. So a red check on a pull request does not by itself stop a merge. A job that fails in the merge queue ejects the pull request, and that does.Wiring only the
pull_requesthalf would have produced a gate that goes red and merges anyway. Trunk and nightly stay skipped —HEADequalsorigin/mainthere, so the diff is empty and the run could only ever pass.Maven property activation has no OR, hence two profiles rather than one condition. Activation reads the runner's own
GITHUB_EVENT_NAME, which is why no workflow file needs to change.Semantics
successCodeslists0and nothing else. Findings (1) and a harness that could not run (2) both fail. A gate nobody has to obey is a report; a gate that reports "clean" without having looked is worse than no gate, because it is indistinguishable from a clean pull request.<timeout>fails the same way, bounding the harness's unshallow git-fetch fallback (a full fetch here is ~1.1 GB / ~20 min).useMavenLoggeris pinnedfalsewith a comment, even though it is the default — that default is the only reason the::errorlines reach the log at column 0 where GitHub can render them. Setting ittruekills every annotation while leaving the build green.Known gap, accepted knowingly
There is no escape hatch. A frontend change that legitimately must add a violation — porting legacy code, an unavoidable third-party shape — cannot merge until the violation is fixed or the gate is switched off repository-wide. No label, no per-file suppression.
The spike measured 0 false positives across its corpus (11 findings across 5 pull requests, all real), so the expected frequency is low, and inventing a bypass before anyone needs one tends to produce the bypass everybody uses. If this bites, say so on #37536.
Reviewer check
Verified locally with the real CI command, on a branch carrying a deliberate
TS7006:BUILD SUCCESS, no gate output at allGITHUB_EVENT_NAME=pull_request::error …dot-utils.ts,line=270,col=35::at column 0, summary table writtenGITHUB_EVENT_NAME=merge_group, violation presentBUILD FAILURE—Failed to execute goal …:exec (strict-gate) … Process exited with an error: 1Activation measured with
help:evaluate -Dexpression=skip.strict.gate:GITHUB_EVENT_NAMEpull_requestfalsemerge_groupfalsepushtruescheduletruetrueThree deliberate deviations from the issue's acceptance list
lint-staged.config.mjshook, and no local hook of any kind.<skip>keys off its ownskip.strict.gate, not${skip.validate}. The issue asks for the latter verbatim. Deliberate: a blocking gate that the documented way to silence its two neighbours also silences is not much of a gate. The flip side is that a plain local./mvnw -Pvalidateruns lint and prettier but not this.Worth recording for whoever picks up a local hook later — the harness compares two committed points, so a commit-time hook would examine the previous commit and report "clean" on the very violation being committed. That was verified, not theorised (
checked 0 project config(s)against an uncommitted change). A local hook has to sit at push time, or the harness needs a notion of staged content. Reasoning preserved inspecs/37536-wire-strict-gate-ci/spec.mdunder C-001/C-002.Out of scope
Checklist
Additional Info
The gate runs — confirmed.
exec:3.0.0:exec (strict-gate)appears at line 1448 of job 104166954128, so the two-profile activation works and the harness is no longer dead code.But this run proves less than that sentence suggests, and one finding from review changed the picture materially. That same run showed the gate had no merge base and was comparing against the tip of
origin/main— which, withmaindeliberately not strict-clean, would have rejected authors for violations they never wrote. Fixed in 61f3fde by resolving the base from the event payload. Details on the review thread.Still owed before merge: a throwaway branch carrying a deliberate
TS7006, confirming exit 1, the annotation rendering inline on the diff, and themerge_groupbehaviour. This branch changes no.ts, so none of that is exercised yet.What lands on the team: from the merge onward, a frontend pull request with a strict violation on a line it wrote will not merge. The spike's rate was ~2 findings per frontend pull request, all of them real.
Follow-up owed on the issue: observed duration and diff size from at least five real pull requests. The gate now sits on the critical path of every frontend merge, so if the real distribution is worse than the corpus suggested (8.4–9.4 s average, 12 s tail), the conversation is about making the harness faster.
Known unrelated redness:
corpus.acceptance.test.mjsneeds network access to the GitHub API and carries machine-dependent timing assertions. It is red offline both before and after this change; the other 10 test files pass with zero failures.🤖 Generated with Claude Code