Add type-strip mode: run Harper directly from .ts sources

.github/DESIGN.md

@@ -28,6 +28,10 @@ Pushes to `main` and release branches run the same staging path, copying into a
 
 Write a fixture's rows in one transaction and await a write only before the step that reads it. Empty a table with `Table.clear()`, or with every delete in one transaction where the table has a full-text index (`clear()` refuses it) or the suite reads the deletes from the audit log (`clear()` writes none). Where a test checks write order (calibration discovery scans decisions by time), keep those writes serial and let the independent ones commit alongside them. Before raising the cap, compare each suite's time in the `lmdb` step with the same suite in `resources (core)`/`resources (indexes)` of the same job, and fix the suite whose LMDB time grew.
 
+## Core CI partitions directories without dropping runtime coverage (`workflows/unit-test.yml`)
+
+The compiled core gate runs `test:unit:main:core`, `test:unit:main:components` and `test:unit:main:server` as separately bounded steps. Components and server each select their directory's `**/*test.*js`; core delegates the unchanged canonical `test:unit:main` with those two directories excluded. The three selections are disjoint and exhaustive, including `.mjs` runtime probes. New files land in their directory's shard, and the full canonical command remains available locally. Cold source/compiled subprocess probes belong in the server step rather than spending the rest of core's failure-reporting window. Each step retains the eight-minute bound; the job backstop exceeds their executable cap sum.
+
 ## A release cherry-pick skips a change its branch already has (`workflows/cherry-pick-patch.yml`, `scripts/change-landed.sh`)
 
 Before picking, the cherry-pick job asks `scripts/change-landed.sh` whether picking the PR onto the release branch could land anything new. The script replays the picked commits onto the release in order with `git merge-tree`, resolving each conflict toward the commit, and the result must be the release's own tree. A conflict then means only that a commit met content a later commit of the PR already wrote. It also merges the PR's net change, `MERGE_BASE..HEAD_SHA` (`MERGE_SHA^1..MERGE_SHA` for a merge-commit merge), into the release, and that must leave the tree unchanged; this covers merge resolutions that no commit carries. Only that answer skips the PR, with no branch and no PR. A change whose file content is partly present, reverted, absorbed by main from another PR, or undone by a later commit of its own picks as before. So does a conflicting change, and so does a check that cannot run. "Unchanged" is decided by Git's merge rules, the same rules the picks themselves run under: a gitlink the release already moved past, or a configured merge driver that keeps the release's side, counts as present for both. Replaying a landed PR one commit at a time re-applies each intermediate commit against its own final content, which conflicts instead of coming out empty. harper#3067 was the second run of a `demilestoned`/`milestoned` pair re-picking harper#3038 onto the v5.3 the first run had just landed it on.

.github/workflows/review-coverage.yml

@@ -122,6 +122,7 @@ jobs:
             dataLayer/backupRepository.ts
             dataLayer/harperBridge/ResourceBridge.ts
             dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetBackup.js
+            dataLayer/harperBridge/lmdbBridge/lmdbMethods/lmdbGetBackup.ts
             dataLayer/restoreMarker.ts
             dataLayer/rocksdbBackup.ts
             dataLayer/schemaDescribe.ts

.github/workflows/unit-test.yml

@@ -29,8 +29,8 @@ jobs:
     runs-on: ubuntu-latest
     # A job that hits this cap is reported as `cancelled`, not `failure`, so it reads as non-red
     # on a required check. Every step below is capped in its own right and this sits above their
-    # sum (55 on the longest leg), so a stall surfaces as a step failure and this stays a backstop.
-    timeout-minutes: 56
+    # sum (72 on Node 22/24, 69 on Node 26), so a stall surfaces as a step failure and this stays a backstop.
+    timeout-minutes: 73
     strategy:
       fail-fast: false
       matrix:
@@ -120,7 +120,17 @@ jobs:
       - name: 'Unit tests: main'
         if: "!cancelled() && steps.setup-harper.outcome == 'success'"
         timeout-minutes: 8
-        run: npm run test:unit:main
+        run: npm run test:unit:main:core
+
+      - name: 'Unit tests: components'
+        if: "!cancelled() && steps.setup-harper.outcome == 'success'"
+        timeout-minutes: 8
+        run: npm run test:unit:main:components
+
+      - name: 'Unit tests: server'
+        if: "!cancelled() && steps.setup-harper.outcome == 'success'"
+        timeout-minutes: 8
+        run: npm run test:unit:main:server
 
       - name: 'Unit tests: apitests'
         if: "!cancelled() && steps.setup-harper.outcome == 'success'"
@@ -142,6 +152,46 @@ jobs:
         timeout-minutes: 6
         run: npm run test:unit:lmdb
 
+  unit-test-source:
+    name: Native Unit Test (Node.js v${{ matrix.node-version }})
+    runs-on: ubuntu-latest
+    timeout-minutes: 39
+    strategy:
+      fail-fast: false
+      matrix:
+        node-version: ${{ (github.event.inputs.node-version == 'all' || github.event.inputs.node-version == '') && fromJSON('["22.18.0", "24.0.0", 22, 24, 26]') || fromJSON(format('[{0}]', github.event.inputs.node-version)) }}
+    steps:
+      - name: Checkout code
+        timeout-minutes: 3
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - name: Setup Node.js
+        timeout-minutes: 5
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: ${{ matrix.node-version }}
+          package-manager-cache: false
+          check-latest: ${{ matrix.node-version == 26 }}
+      - name: Install dependencies
+        timeout-minutes: 6
+        run: npm ci --ignore-scripts
+      - name: Verify no compiled distribution
+        timeout-minutes: 1
+        run: test ! -d dist
+      - name: Check unit harness types
+        timeout-minutes: 2
+        run: npm run test:types:unit
+      - name: Native core units
+        timeout-minutes: 8
+        run: npm run test:unit:typestrip
+      - name: Native resource units (core)
+        if: '!cancelled()'
+        timeout-minutes: 8
+        run: npm run test:unit:typestrip:resources:core
+      - name: Native resource units (indexes)
+        if: '!cancelled()'
+        timeout-minutes: 5
+        run: npm run test:unit:typestrip:resources:indexes
+
   # Windows had no unit-level coverage at all before this job: every job above pins
   # ubuntu-latest, and integration-tests.yml was the only workflow touching Windows.
   # Platform-gated code and the tests written to prove it therefore never executed.

AGENTS.md

@@ -46,7 +46,7 @@ Run a single test file directly:
 npx mocha unitTests/resources/mytest.js
 ```
 
-TypeScript is stripped at runtime via `--conditions=typestrip` (Node.js native type stripping) — no compilation required for development. Use `npm run test:unit:typestrip` to run tests with this mode.
+TypeScript is stripped at runtime via `--conditions=typestrip` (Node.js native type stripping). `npm run test:unit:typestrip` runs the source-compatible core subset without compilation; export-rewriting and compiled-artifact fixtures retain `#js`. Run `npm run build` followed by `npm run test:unit:typestrip:main` for the complete core selection in separate source and compiled processes.
 
 **Unit-test isolation:** every mocha run gets its own per-PID root under `unitTests/envDir/<pid>` — config file, log, keys, and every database including `system` (`unitTests/mocha.init.js` builds it before any test loads; `setupTestDBPath()` re-asserts the paths). A suite that needs the system tables to exist seeds them with `ensureSystemTables()` from `unitTests/testUtils.js` (system tables, a super_user role with an `admin` user, self-signed certs — what an install would create). After each test file, `unitTests/perFileDatabaseTeardown.js` drops every database that file created, and on RocksDB also removes one it closed but left on disk (the module header says why). So a test file must not depend on a database another file created, and must not create one in a root-level hook or test (outside any `describe`); `system`, configured databases and databases that existed before the file started are kept. Unit runs therefore stay out of an installed Harper root, can run while a local Harper is running, and work on a machine with no Harper install at all. Known carve-outs: `test:unit:apitests` still stops a running Harper first, for port reasons only (a wildcard-bound instance holds `*:9925/9926/1883`, starving the loopback-address pool); and the few test files that clear mocha.init.js's `ROOTPATH` export to exercise boot-props resolution fall back to the installed root's resolution inside their scope.
 

CONTRIBUTING.md

@@ -17,9 +17,29 @@ Build the project using `npm run build` or `npm run build:watch` to automaticall
 
 Run integration tests using `npm run test:integration:all`. For targeted runs or additional options, use `npm run test:integration -- <pattern>`. Make sure to read the [integration test instructions](./integrationTests/README.md) for setup requirements (particularly the loopback address configuration).
 
-Run unit tests using `npm run test:unit <unit-test-file>` or `npm run test:unit:all`, but make sure to build the project first since unit tests depend on the built source files.
+Run a source-compatible test directly, without building Harper:
 
-> Unit tests currently use [Mocha](https://mochajs.org/) as the test runner, but since they are implemented in TypeScript and are sometimes executing TypeScript source code, it also uses [TSX](https://tsx.is/) for compilation and execution. The npm script `test:unit` sets the appropriate env vars and mocha configuration file. Make sure that the `TSX_TSCONFIG_PATH` environment variable points to the correct `tsconfig.json` file for the unit tests (i.e. `./unitTests/tsconfig.json`) and not the root-level `tsconfig.json`.
+```bash
+npm run test:unit:typestrip -- unitTests/components/deploymentRecorder.test.js
+```
+
+`#src` selects the source tree in this lane; retained JavaScript sources use explicit mappings or a `.js` suffix. `npm run test:unit:typestrip` runs the source-compatible part of the core gate. Native resource tests use `npm run test:unit:typestrip:resources`. Source conditions are inherited by workers and spawned fixtures.
+
+Legacy tests that rewire exports, replace module namespaces or manipulate their export cache retain `#js` and need compiled CommonJS. Build after changing the code they test, then run them with ordinary Mocha:
+
+```bash
+npm run build
+npm run test:unit -- unitTests/config/configUtils.test.js
+npm run test:unit:typestrip:main
+```
+
+The combined `:typestrip:main` command covers the same selection as `test:unit:main` in two sequential processes. Each has its own isolated configuration and databases. `--list` on `node unitTests/runTypeStripTests.mts` shows the partition. `:typestrip:all` selects all `*test.*js` suites, including the API and legacy storage suites; stop a conflicting server before API tests as for the ordinary API gate.
+
+Use the project-installed Mocha through the npm scripts or `npx`; completion tracking patches that copy.
+
+A source process denies `#js`, and the harness fails a source run that loaded this checkout's compiled core, checking at file boundaries and final completion. Route an existing mutable-export suite to the compiled lane instead of mixing trees. The combined launcher rejects a globally enabled `typestrip` condition; let it choose each lane. Ordinary compiled CI still runs all tests, while native CI never builds `dist`.
+
+The launcher runs each lane in one process, so use ordinary Mocha for watch or parallel mode. Custom configurations must require `unitTests/mocha.init.js` for completion tracking. Mixed selections with global `--file` setup run compiled to preserve ordering. File reporters and external reporters must run separately per lane to avoid overwriting output. Ctrl-C removes the launcher's temporary configuration; a force-killed process can leave that directory and its isolated database root. The existing stale-root sweep reclaims only the database root. `.mocharc.json` sets Mocha's per-test timeout to zero (no limit). CI bounds each step; use `--timeout` for local per-test limits. The full selection needs a current build for legacy and artifact checks.
 
 ## Code Formatting & Linting