Support TypeScript 7.1 API - #1704
Draft
johnnyreilly with Copilot wants to merge 124 commits into
Draft
Conversation
Copilot created this pull request from a session on behalf of
johnnyreilly
July 26, 2026 09:47
View session
Member
|
@copilot update ts-loader/.github/workflows/push.yml Line 52 in 53082fb ts-loader/.github/workflows/push.yml Line 84 in 53082fb We expect execution-tests to pass after this change |
Contributor
Author
Updated in |
Member
|
@copilot great work! We would like the following to succeed:
Remember backwards compatibility is not required - feel free to make changes |
Contributor
Author
Verified on |
Member
|
@copilot we have explictly removed the historic typescript version from dependencies and moved to use 7.1 directly. This has broken compilation. Please fix it. You will likely need to use new APIs in TypeScript 7 to achieve this. No cheating. |
Contributor
Author
Updated in |
Member
|
@copilot make |
Contributor
Author
Addressed in |
updateSnapshot passed { invalidateAll: true } unconditionally on every
single file compiled, forcing the native API to discard its caches and
rescan disk once per file rather than once per build - real watch-mode
wiring (a hook tied to webpack's own build/rebuild boundary) didn't
exist anywhere in the tsgo rewrite.
Add TypeScriptInstance.pendingInvalidation, consumed by the first
updateSnapshot call of a build and re-armed via a compiler.hooks.compile
tap (fires once per build/watch-rebuild in both webpack 4 and 5), so
only that first call forces the rescan; every other file compiled in
the same build reuses the snapshot it already refreshed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…heck findTransitiveDependants rebuilt a fresh O(n) Set of every project file's name inside getDirectResolvedImports on every single call, and called it once per project file for every file compiled - roughly O(n^3) for a full build. The currently-compiling file's own imports were also recomputed a second time (once via registerResolvedImportDependencies, again inside the BFS). - comparableSourceFileNames is now built once per file compiled (passed down to callers) instead of once per candidate file inside the BFS. - Each project file's resolved-imports list is memoized in a new TypeScriptInstance.directImportsCache, reused across every file compiled in the same build via getCachedDirectResolvedImports. The currently-compiling file always recomputes fresh (never trusts a stale entry) and writes its result back, so other files' dependant searches - and the file's own subsequent BFS lookup - see it too, incidentally also fixing the double computation. - The cache is cleared whenever pendingInvalidation forces a real rescan (see the prior watch-mode fix), keeping it correct across builds. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
registerResolvedImportDependencies wrote cache entries keyed by apiFileName (OS-native, from webpack), while findTransitiveDependants/getCachedDirectResolvedImports read/wrote entries keyed by program.getSourceFileNames() (forward-slash- normalized). On Windows those are different strings for the same file, so an entry written by one path could never be found by the other - silently defeating the memoization exactly where it mattered (the currently-compiled file's own entry). Key by FilePathKey via each instance's ResolvedPathCache instead, matching how instance.files/pendingDiagnostics/pendingDeclarationFiles already canonicalize path spelling and case-sensitivity. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
registerTypeScriptDependencies iterated program.getSourceFileNames() (every file in the program) on every single compiled file just to find .d.ts files to register as dependencies - O(n) per file, O(n^2) over a build, even though that set only changes when the program's file set actually does. Add getProjectDtsFileNames, memoized per project (primary or a synthetic one-off project for an orphan file) in a new TypeScriptInstance.projectDtsFileNamesCache, cleared alongside directImportsCache whenever pendingInvalidation forces a real rescan. Keyed by FilePathKey (via ResolvedPathCache) for consistency with directImportsCache, though project config paths are already stable, single-sourced strings in this codebase. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ensureSyntheticConfigForFile created and permanently tracked one synthetic tsconfig + open project per distinct orphan file (e.g. every allowTsInNodeModules file ever compiled), with no eviction - each held its own ref-counted project open on the API side, so a long watch session touching many distinct orphan files leaked memory/state proportional to all of them, unbounded. Cap it at 20 (maxOrphanFileProjects), evicting the least-recently-used entry - tracked via Map insertion order, bumped on reuse - once the cap is hit. The evicted project's closeProjects is threaded through updateSnapshot to actually release its ref-counted open on the API side, and removed from openedProjectPaths so revisiting that file later reopens it fresh rather than treating it as still-open. Verified with a standalone scratch reproduction (22 distinct orphan files under allowTsInNodeModules): exactly 2 evictions occurred against the cap of 20, the build succeeded with no errors, and editing a previously-evicted file was correctly picked up on the next rebuild. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two real bugs, same class as the earlier directImportsCache fix: - syntheticConfigContents's readFile/fileExists lookups used toComparablePath (slash-only normalization), while the sibling files Map right next to it already used resolvedPathCache (slash *and* case normalization) - a case-insensitive-filesystem spelling mismatch would silently miss. - syntheticConfigFiles was keyed by raw fileName. Two importers spelling/casing the same orphan file differently would be treated as distinct files, each spawning its own redundant synthetic project - undermining the LRU cap added previously. configFilePath and openedProjectPaths are converted too for consistency, though their values were already single-sourced and stable in practice. Added TypeScriptInstance.resolvedPathCache so functions that only receive TypeScriptInstance (not the outer TSInstance) - openPrimaryProject, prepareSnapshotForFile, ensureSyntheticConfigForFile, updateSnapshot - can canonicalize a path without threading it through as a separate parameter. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Use interface instead of type for the plain object shape TypeScriptApiModule, matching AGENTS.md's convention. - Extract reportConfigFileParsingErrors, deduplicating the ~20-line broken-tsconfig error-reporting block shared by getTypeScriptEmit and getTranspileOnlyEmit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TypeScriptInstance.resolvedFilePathCache duplicated the exact same function reference already stored on the owning TSInstance's own resolvedPathCache field. Drop the stored field; thread it as an explicit parameter through prepareSnapshotForFile/ensureSyntheticConfigForFile instead, sourced from instance.resolvedPathCache at the getTypeScriptEmit call site - matching the convention already used by getProjectDtsFileNames, registerResolvedImportDependencies, getCachedDirectResolvedImports, and findTransitiveDependants. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…opilot/implement-new-tsgo-api-support
The tsgo sync API spawns a native child process per ts-loader instance, making a "cold build" iteration ~10-20x more expensive than the classic API's cheap in-process instantiation; touching a widely-imported (hub) file is similarly ~35x more expensive per incremental rebuild due to per-dependant recheck round trips over the sync RPC channel. The fixed iteration counts (tuned for the classic API's cost profile) made the cold-typeCheck and hub-touch scenarios alone take ~28 minutes combined, blowing the 20-minute CI job timeout before a single scenario finished. Cap each scenario's wall-clock time in run-side.mts instead of guessing a smaller fixed iteration count, so cheap scenarios keep their full sample size while expensive ones stop once they've collected enough measured samples. Confirmed locally: full default run (300 files) went from never finishing to completing in ~4 minutes.
Benchmark (Ubuntu)
PR branch = |
Benchmark (Windows)
PR branch = |
…per file Profiling the benchmark's slowness (tsgo branch showing 10-20x worse numbers than the classic branch, contrary to tsgo's own "faster compiler" characteristic) traced 83-91% of both cold-build and hub-touch-rebuild time to recheckTransitiveDependants: it ran once per file webpack compiled rather than once per build, and each call does a full O(project size) dependant search plus two diagnostic calls per dependant found. For a wide-fanout change (a file most of the project depends on) that's close to O(n²) work in a single build. The raw tsgo API itself opens a 300-file project and double-diagnoses every file in ~25ms - it's not the bottleneck. Batches the recheck into one pass per build instead: getTypeScriptEmit now just records which files it compiled (changedFilesThisBuild), and a new recheckAllTransitiveDependants runs once from the existing postCompile hook, searching dependants of the whole changed-file set in a single pass. Getting this right needed two follow-up fixes surfaced by comparison tests, both around same-build ordering: - A changed file can itself import another changed file compiled later in the same build (e.g. an entry file and the dependency it just changed). Its own diagnostics may have been computed before that other file's compile updated the shared file cache the API's readFile override serves from. findTransitiveDependants no longer excludes changed files from being found as dependants of each other, so this gets caught and rechecked too. - The recheck's own snapshot needs a forced full rescan (pendingInvalidation = true) rather than reusing the arbitrary anchor file's incremental view, otherwise it can still read a stale copy of a same-build sibling. Only costs one extra rescan per build (not per file), so it's affordable now. Measured on a synthetic 300-file fixture: cold build ~9.1s -> ~1.0s, hub-touch incremental rebuild ~4.2s -> ~300-380ms. Full comparison-test suite (including all watch-mode tests) still passes.
…opilot/implement-new-tsgo-api-support
…opilot/implement-new-tsgo-api-support
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copilot didn't write this - I did! The below can also be found in the CHANGELOG.md.
This is a ground-up rewrite of ts-loader's compilation engine. Instead of driving TypeScript's classic
LanguageService/Program/ watch APIs, ts-loader now compiles exclusively through TypeScript's new nativetypescript/unstable/syncAPI (the tsgo-powered engine) - the legacy compiler API integration has been removed entirely.Not supported yet
getCustomTransformers,resolveModuleNameandresolveTypeReferenceDirectiveare still accepted for backwards compatibility but are now inert - the native API doesn't expose equivalent extension points, so custom transformers and custom module/type-reference resolution are no longer applied. It is possible that the API will support these in future, and so the options have been left in place for now, but they will be removed if the API never exposes them.Breaking changes:
nextprerelease that exposes this native API ahead of a stable 7.1 release.compileroption must now resolve to a package exposing a<compiler>/unstable/syncentry point (the TypeScript native API). Drop-in classic-API compilers (e.g.ttypescript) are no longer supported.compilerOptionsloader option; the native API resolves a project's compiler options purely from its on-disk tsconfig.json, with no per-loader-instance override hook. Set compiler options intsconfig.jsoninstead.contextloader option; the native API always resolves relative paths against the config file's own directory, with no basePath override exposed to let a tsconfig live outside the project root.happyPackModeloader option - HappyPack / thread-loader based parallelisation is no longer supported this way.experimentalFileCachingloader option; there's no equivalent under the native API's own caching model.experimentalWatchApiloader option, now that the native API supersedes TypeScript's classic watch API.onlyCompileBundledFilesloader option; the native API always resolves a project's own root files and offers no hook to restrict them to what webpack actually bundles.errorFormatter'scolorsargument is now a smallpicocolors-backed helper object instead of achalkinstance;chalkhas been dropped as a runtime dependency in favour ofpicocolors.