Build harper-pro's dist through core's comment-free, Latin-1 emit, cutting about 10 MB from each worker's heap

[kriszyp]

⊙ Problem

harper-pro's npm run build compiles core/** into dist/core with plain tsc, so it never runs core's production emit from Reduce each thread's base JavaScript heap by about 40%. Every Harper Pro worker therefore keeps all of core's comments in module source on its heap, and stores any module containing a — or → as UTF-16. Measured on an idle worker: harper:core module source is 6.8 M chars (dist/core/resources/Table.js alone has 1,575 comment lines).

❓ Your call: harper-pro calls core's script (core/build-tools/build-dist.mjs --project tsconfig.json) instead of keeping its own copy. This ties Pro's build to that script's path and CLI flag in core; the alternative was a Pro-side driver importing only the exported transformer, which duplicates core's ~40-line build driver. Reverting is a one-line package.json change.

💡 Solution

npm run build now runs core's emitter against Pro's own tsconfig, so all of dist/ (Pro code and dist/core) is comment-free and Latin-1, and the build fails if a non-Latin-1 character survives. npm run build:watch stays plain tsc --watch (comments kept, development only). Packaging no longer ignores a failed build, so that Latin-1 check can actually stop a release.

Idle worker, GC'd process.memoryUsage().heapUsed (same core, only the emit differs):

Build	heapUsed	harper:core source	harper:replication source
plain tsc (before)	51.9 MB	6,829 K chars	1,075 K chars
core build-dist.mjs (after)	42.1 MB	4,692 K chars	599 K chars
today's main (pinned core, plain tsc), for reference	64.1 MB	6,832 K chars	1,129 K chars

external_script_source_size stayed at 4.3 MB in every run; that is source held off the V8 heap (most likely Node's own built-ins), so the saving shows in heapUsed instead.

🔧 Changes

• package.json: build runs node core/build-tools/build-dist.mjs --project tsconfig.json; build:watch calls tsc --project tsconfig.json --watch --incremental directly, since build no longer accepts tsc flags.
• core: submodule moves to the head of Reduce each thread's base JavaScript heap by about 40%, which adds build-dist.mjs and its --project option (plus that PR's lazy dependency loading and source-map default).
• build-tools/DESIGN.md, DESIGN.md: design note on why Pro builds through core's emitter, indexed from the root.
• AGENTS.md: build command descriptions.

build-tools/build-pro.sh is no longer part of this diff: #981 landed first and already made the same rm -rf dist / strict npm run build change (dropping || true) and removed the shrinkwrap step this PR's rm -f npm-shrinkwrap.json guard was for. Rebasing onto main absorbed this PR's build-pro.sh hunk entirely; "packaging no longer tolerates a failed build" ships from #981 instead of from here. build-tools/DESIGN.md now carries both PRs' notes under separate headings, as predicted.

The core pointer has been re-bumped to merged harper#3072's commit, which is also HarperFast/harper main's current tip, during this rebase onto main.

✅ Verification

• End-to-end route: live smoke with recorded measurements. Built dist each way, started Harper Pro with 2 workers and per-thread inspectors (threads.debug.startingPort=9340), forced GC over CDP and read process.memoryUsage(), V8 code statistics and the loaded-script inventory per thread (table above; each config run twice, both workers within 0.1 MB). After the build, no dist/**/*.js contains a character above U+00FF, and dist/core/resources/Table.js has one comment line left (its sourceMappingURL). npm run build:watch starts with 0 errors.
• Gates (local, macOS): npm run build, npm run typecheck, npm run lint:required pass. test:unit: 1,628 passing, 4 failing; the same 4 fail on a plain-tsc build of the same tree (/proc, macOS symlink paths, git fixtures). test:integration:all: 218 passed, 4 failed, 2 cancelled of 239. analytics, isolatedApplicationReplication and replicateFalseFullCopy fail identically on the plain-tsc build; deleteEchoRun failed once under full-suite load and passed 3/3 when rerun alone. CI is authoritative.
• prettier --check is clean on every changed file.
• Rebase onto main (round 1, 1464ea6 → 76cc842): resolved conflicts in DESIGN.md, build-tools/DESIGN.md (both PRs' notes kept) and build-tools/build-pro.sh (dropped — redundant with Publish harper-pro with a locked JavaScript dependency bundle instead of a shrinkwrap #981, now on main); re-bumped core to merged harper#3072's commit. CI then failed Unit Tests because sendLoopYield.test.mjs located code by searching for a comment the new comment-free build strips; fixed by anchoring on a code token instead.
• Rebase onto main (round 2, 76cc842 → 60a4c80): no textual conflicts — main's own core sync (chore: Sync Core #1039) landed a newer, descendant commit of the core pin this PR carried, so Git's submodule-aware merge auto-resolved the gitlink forward to main's pin (3fd9ae9d, confirmed to still contain harper#3072). No other overlapping file changed on both sides. npm run build and npm run typecheck pass on the rebased tree; dist/**/*.js has zero bytes above U+00FF; full test:unit passes, 1793/0 failing (2 pre-existing pending). Independent re-review (codex + gemini + cursor-composer + harper-domain adjudication): LGTM, 0 actionable findings — the 2 surviving nits (build:watch/production divergence, the test's first-occurrence anchor) are both pre-existing, already-documented tradeoffs, not fixed.

The core submodule pointer now matches HarperFast/harper's own main tip exactly (no longer ahead of it), so this PR no longer carries an active dependency on an unmerged companion.

_{🤖 Generated by Claude Code (Claude Opus 5.5); posted via @kriszyp.}

Release target: v5.4 — milestoned for v5.4; the build-tooling it touches also exists on v5.3, but no backport PR is linked, so this stays main-only
Related PRs: #956 overlaps (also moves the core submodule pointer), #972 overlaps (also edits AGENTS.md and the DESIGN.md index), #981 overlaps (makes the same rm -rf dist / no || true change to build-pro.sh, drops the shrinkwrap, and also creates build-tools/DESIGN.md), #983 overlaps (also moves the core submodule pointer), #985 independent (merged core sync), #987 independent (merged), #995 overlaps (its current head pins core at the same 2f2b428b this PR bumps to; whichever lands second is a no-op pointer change), #997 overlaps (also touches build-tools/build-pro.sh and the core pointer), #1035 independent (merged Sync Core; supplied the previous core pin this PR moves forward from), #1014 independent (merged backport of #981's bundling change), #998 independent (merged), #1001 independent (merged), #996 independent (merged), #999 independent (merged)

Complexity: easy

_{Review-Coverage: authored=claude; ran=gemini,cursor-composer,codex; adjudicated=domain; declined=cursor-grok,cursor-kimi,cursor-muse; rounds=3; full=2 @ 60a4c80}

_{Review-Attention: skim ~2m (decisions: watch-emit-parity, build-coupled-to-core-path) @ 60a4c80}

[gemini-code-assist]

Code Review

This pull request updates the build process to use the core's custom build tool (build-dist.mjs) for production builds, ensuring that the emitted JavaScript is comment-free and has non-Latin-1 literals escaped to optimize V8 heap usage. Additionally, build-pro.sh is updated to clean the dist directory and fail on build errors instead of ignoring them, and the package.json scripts and documentation are updated accordingly. There are no review comments, and I have no feedback to provide.