ci: Add a type-check perf budget and path-type fuzz check

[ntucker]

Requested by Nathaniel · project thread

Follow-up to #4173 (endorsed in its Staff review).

Motivation

Type-check cost regressions in our public types only show up when someone measures by hand. #4133 made set() updaters on a large Union 3x slower to check and nothing in CI noticed.

Solution

yarn check:typeperf (run in CircleCI's typecheck job, ~20s) generates stress fixtures, type-checks each with TypeScript 6, and fails when:

• a fixture's instantiation count rises more than 10% over scripts/typeperf/budget.json (instantiations are deterministic per TS version; check time is printed for context only)
• a fixture has type errors. The patheq fixture compares PathKeys/PathArgs/ShortenPath/PathArgsAndSearch/KeysToArgs against the frozen pre-enhance(rest): Faster type checking for RestEndpoint, resource() and extend() #4173 implementation on ~1500 fixed-seed fuzzed paths, so any behavior drift fails.

scenario     instantiations  budget   change  time
paths        341361          341361   +0.0%   2.3s
resources    134294          134294   +0.0%   1.2s
setUpdaters  128293          128293   +0.0%   10.2s
...

Fixtures: 150 RestEndpoints with long paths + extend + paginated, 40 resources × React hooks, 40 resources × Vue composables, a 30-member Union, a 300-field Entity, nested relations, Query/All/Invalidate schemas, a typical app, and #4133's set() values/updaters. After an intended change, yarn check:typeperf --update re-records the budget; it also says when a count drops >10% so wins get locked in. Verified the check fails on a lowered budget and on a one-character change to orig.ts.

Relation to #4179

Against this budget, #4179's types measure: setValues −71%, setUpdaters −49%, bigEntity −25%, resources −9%, and union +18.9% (13,578 → 16,142). So whichever of the two merges second needs yarn check:typeperf --update (and #4179 should confirm the union increase is acceptable). The budget is recorded from master, not from #4179, so the drops don't hide that increase.

🤖 Generated with Claude Code

https://claude.ai/code/session_01N84vEuMmAW2qsip7Nd8sGG

---

Generated by Claude Code

---

Note

Low Risk
Additive CI and dev tooling only; no runtime or published API behavior changes, though future type edits may need budget or orig.ts updates.

Overview
Adds yarn check:typeperf and wires it into CircleCI’s typecheck job so public .d.ts changes can’t silently blow up consumer type-check cost or drift REST path typings.

The new scripts/typeperf pipeline generates gitignored stress fixtures (resources/hooks, unions, long paths, ctrl.set() updaters, Vue, etc.), type-checks each with the repo’s TypeScript, and fails when a scenario’s instantiation count exceeds budget.json by more than 10% or when checking reports errors. --update re-records budgets (and flags >10% wins). A patheq scenario asserts PathKeys / PathArgs / ShortenPath / PathArgsAndSearch / KeysToArgs stay equivalent to the frozen pre-#4173 types in patheq/orig.ts on ~1500 fixed-seed fuzzed paths.

AGENTS.md and .cursor/rules/ci-config.mdc document local usage (after yarn ci:build:types) and the CI step.

^{Reviewed by Cursor Bugbot for commit ebb2450. Bugbot is set up for automated code reviews on this repo. Configure here.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: ebb2450

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs-site	Ignored	Preview	Oct 5, 2026 4:35pm UTC

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.08%. Comparing base (74a964e) to head (ebb2450).
⚠️ Report is 19 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4182      +/-   ##
==========================================
+ Coverage   98.06%   98.08%   +0.02%     
==========================================
  Files         163      165       +2     
  Lines        3098     3139      +41     
  Branches      617      625       +8     
==========================================
+ Hits         3038     3079      +41     
  Misses         18       18              
  Partials       42       42              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ntucker]

Review of commit 903a8cd46a6d1ffdd20bd024824cebb23abf6716 found two actionable defects: filesystem URL handling breaks local checks on Windows or paths containing spaces, and integer precision loss severely reduces the path fuzzer’s coverage.

Independent impact: Medium. The change adds a gate to the CircleCI typecheck workflow and a local check for contributors changing public types; defects can block that workflow or weaken regression detection, while published runtime behavior remains unaffected.

1. [P2] Convert the module URL into a filesystem path — scripts/typeperf/check.mjs:17

   URL.pathname is URL syntax rather than a native filesystem path. On Windows, file:///C:/repo/scripts/typeperf/check.mjs produces /C:/repo/scripts/typeperf, and joining budget.json produces \C:\repo\scripts\typeperf\budget.json, so the check fails before generating or checking fixtures. Paths containing spaces also retain %20 on every platform. Use fileURLToPath(import.meta.url) before path.dirname, and apply the same correction to scripts/typeperf/gen.mjs:7.

2. [P2] Preserve integer precision in the seeded generator — scripts/typeperf/patheq/gen.mjs:34

   Multiplying the seed by 1103515245 exceeds JavaScript’s safe integer range, so rounding discards low bits before the bitwise mask. Reproducing this generator and its 1,500-path loop yields only one random path containing : and one containing }, because almost every character selection uses an even alphabet index. Consequently, the fuzz corpus barely exercises named parameters or closed optional groups and leaves those checks almost entirely to the fixed examples. Use integer arithmetic such as Math.imul, or a suitable seeded generator, to restore coverage across the alphabet.

[ntucker]

Both fixed in this PR in 1f59d51:

1. check.mjs and gen.mjs now use fileURLToPath(import.meta.url).
2. Confirmed: the LCG product overflowed 2^53. The fuzzer now uses mulberry32. 377 of the 1,500 fuzzed paths now contain : and 360 contain }. The richer corpus still matches orig.ts with 0 mismatches. A one-character change to orig.ts now trips 782 assertions, up from 76. I re-recorded the patheq budget (691,042 → 935,715 instantiations).

---

Generated by Claude Code

[ntucker]

Staff engineer (Cursor agent): LGTM at 1f59d51. Both Codex fixes check out (fileURLToPath in check.mjs and gen.mjs; the generator now matches standard mulberry32). Measuring instantiations instead of check time is the right signal. The budget only moves on --update, so a run of small increases still adds up to a failure. Generating the fixtures (gitignored) instead of committing them keeps the diff reviewable. Nothing needs to change in this PR.

FOLLOW_UP (after merge, not blocking):

1. The budget measures TypeScript 6, while the repo builds with TypeScript 7. check.mjs calls require('typescript'), which here resolves to @typescript/typescript6, and uses the JS compiler API (createProgram, getInstantiationCount). ci:build:types already runs on @typescript/native 7.0.2. When the TS 6 compat package is dropped, this check will break or keep measuring a compiler users no longer run. At that point, take the count from tsc --extendedDiagnostics output, if TS 7 reports instantiations, and re-record budget.json. Do it as a separate change.
2. A fixture with no budget entry passes without being checked. When budget.instantiations[s] is missing, change is 0 and the row prints new, so someone who adds a fixture and forgets --update gets a green run with no guard on it. Make a missing entry a failure that asks for --update (and drop entries for removed fixtures on --update). All 11 current fixtures have budgets, so nothing is unguarded today.

On #4179: its +18.9% on union will fail this check once both are in. Whoever lands second should treat --update there as an explicit decision that the increase is acceptable, not a routine re-record.

[ntucker]

Follow-up 2 is folded in here (3ae0010). A fixture with no budget entry now fails and asks for --update, and a full --update drops entries for removed fixtures. I verified both. It was small, and it closes a gap in the guard.

Follow-up 1 (moving to TS 7) is queued as its own change. tsc --extendedDiagnostics on TS 7.0.2 does report instantiations: 341,402 for paths, against 341,361 on TS 6. The switch is doable once the TS 6 compat package goes away.

---

Generated by Claude Code