Skip to content

ci: Measure the type-check budget with TypeScript 7 - #4192

Merged
ntucker merged 1 commit into
masterfrom
claude/typeperf-ts7-rc913v
Oct 5, 2026
Merged

ntucker merged 1 commit into
masterfrom
claude/typeperf-ts7-rc913v

Conversation

@ntucker

@ntucker ntucker commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Requested by Nathaniel · project thread

Follow-up to #4182.

Motivation

Before: yarn check:typeperf measured instantiations with the TypeScript 6 JS API (require('typescript') resolves to @typescript/typescript6), while the repo builds and type-checks with TypeScript 7 (@typescript/native).

After: it measures with the same TypeScript 7 tsc the repo builds with, so the budget tracks the compiler users and CI actually run.

Solution

  • check.mjs spawns the repo's tsc (the bin yarn puts on PATH) with --extendedDiagnostics --pretty false per fixture and reads the Instantiations: and Total time: lines. A non-zero exit fails the fixture and prints the first 10 errors; output with no instantiation count fails too. The version comes from tsc --version.
  • Windows support is kept: the shell is used only on win32, and the tsconfig path is passed relative to the repo root so a checkout path with spaces can't be split. The "has no budget" failure from ci: Add a type-check perf budget and path-type fuzz check #4182 is unchanged.
  • budget.json is a new TypeScript 7.0.2 baseline, re-recorded with --update. These numbers are not comparable to the old TS 6 ones. TS 7 counts are within 4% of TS 6 for every fixture except schemas (99,251 → 109,095, +9.9%). That's a compiler difference, not a change to our types: master's types are unchanged in this PR.
  • Counts are deterministic across runs (verified 3× per fixture, and with --checkers 1 / --singleThreaded, since each fixture is one source file).
  • The full check runs in about 12s locally (each TS 7 fixture checks roughly 2.5× faster than TS 6).

Validated: yarn check:typeperf passes; injecting a type error into a fixture fails with tsc exited with 1, 1 type errors and the error line; deleting a fixture's budget fails with has no budget.

Open questions

Running fixtures in parallel would cut the check to about 5–6s on 4 cores without changing counts, but makes the printed times noisier. I left it sequential for now.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QFo5AAtZp18HcwiXBqBDEZ


Generated by Claude Code


Note

Low Risk
Changes only the typeperf CI script and recorded budgets; library types and runtime behavior are untouched.

Overview
yarn check:typeperf now measures instantiation counts with the same TypeScript 7 tsc the repo builds with (@typescript/native), instead of the TypeScript 6 programmatic API.

check.mjs spawns tsc per fixture with --extendedDiagnostics, parses Instantiations and Total time from stdout, and records the compiler version from tsc --version. Non-zero exits and missing instantiation output fail the check; Windows still uses a shell only on win32 and passes a relative tsconfig path.

budget.json is re-recorded for TypeScript 7.0.2 — counts are not comparable to the old TS 6 baseline. The README notes that compiler upgrades require --update when counts shift.

Reviewed by Cursor Bugbot for commit 2ee0f08. Bugbot is set up for automated code reviews on this repo. Configure here.

check:typeperf used the TS 6 JS API (require('typescript') resolves to
@typescript/typescript6) while the repo builds with @typescript/native 7.
It now spawns the repo's tsc with --extendedDiagnostics and reads the
Instantiations line, failing on a non-zero exit. budget.json is
re-recorded as a TypeScript 7.0.2 baseline.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QFo5AAtZp18HcwiXBqBDEZ
@changeset-bot

changeset-bot Bot commented Oct 5, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 2ee0f08

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ntucker ntucker self-assigned this Oct 5, 2026
@vercel

vercel Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs-site Ready Ready Preview Oct 5, 2026 6:28pm UTC

Request Review

@codecov

codecov Bot commented Oct 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.09%. Comparing base (97b3016) to head (2ee0f08).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #4192   +/-   ##
=======================================
  Coverage   98.09%   98.09%           
=======================================
  Files         165      165           
  Lines        3143     3143           
  Branches      625      625           
=======================================
  Hits         3083     3083           
  Misses         18       18           
  Partials       42       42           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ntucker
ntucker merged commit b492e0b into master Oct 5, 2026
22 checks passed
@ntucker
ntucker deleted the claude/typeperf-ts7-rc913v branch October 5, 2026 18:46

This branch was successfully deployed

1 active deployment
Preview — 2ee0f080 Deployed Oct 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants