[kriszyp]

⊙ Problem

During first-time install, the installer created the system database before composing harper-config.yaml. It resolved only the CLI and legacy STORAGE_ENGINE override in memory, while workers later read the config file from disk. Setting the storage engine or RocksDB compression via HARPER_DEFAULT_CONFIG, HARPER_SET_CONFIG, STORAGE_* environment variables, or an HDB_CONFIG template could leave system on RocksDB while workers opened LMDB. HDB_CONFIG installs also skipped applying CLI and environment overrides to the file.

🤔 Should HDB_CONFIG templates also receive environment, CLI, and default backfills during install?

Yes. Operators pass partial HDB_CONFIG templates alongside CLI flags or HARPER_SET_CONFIG overrides, and expect the installer and workers to share one complete config. Backfilling missing defaults and applying layered overrides keeps HDB_CONFIG consistent with standard installs. Restoring the pre-install file on rollback makes it reversible.

💡 Solution

• Compose the full install configuration (HARPER_DEFAULT_CONFIG, HDB_CONFIG template, CLI flags, STORAGE_* env vars, and HARPER_SET_CONFIG) in memory before mountHdb(), and validate the staged configuration before creating system tables.
• Stage the composed config onto the active in-memory config without recording sticky worker overrides, then write that exact document to disk after certificate generation.
• Roll back only files and directories created during the failed install attempt, restoring any pre-existing HDB_CONFIG file on failure.

⚖️ Alternatives

• Only forward createConfigFile() storage settings into envManager before createTables() — leaves HDB_CONFIG template installs broken because main skipped createConfigFile() when HDB_CONFIG existed, so CLI and env overrides never reached disk.
• Write harper-config.yaml beside an existing HDB_CONFIG instead of updating HDB_CONFIG in place — diverges from keys.updateConfigCert() on main, which already rewrites HDB_CONFIG in place, and changes which file harper-boot.json points to.

🤔 Should install update a pre-existing HDB_CONFIG file in place or write a separate harper-config.yaml?

We update HDB_CONFIG in place and restore its original bytes on rollback to match keys.updateConfigCert() on main and keep harper-boot.json pointed at the operator's path. Writing a sibling harper-config.yaml instead is a local change in installer.ts if we prefer treating HDB_CONFIG as read-only.

🔧 Changes

• config/configUtils.ts: Extracts composeInstallConfig(), stageInstallConfig(), validateStagedInstallConfig(), and writeStagedInstallConfig(), backfills missing defaults into HDB_CONFIG templates and tracks authored template leaf paths via backfillMissingDefaults() and collectLeafDotPaths() while keeping createConfigFile(args, skipFsValidation) compatible with existing callers.
• config/harperConfigEnvVars.ts: Extends filterArgsAgainstRuntimeConfig() to filter individual env vars and legacy aliases against HARPER_CONFIG, checks ancestor and descendant path sources in hasRespectedSource(), and accepts userPaths in prepareRuntimeEnvConfig() to seed 'user' sources from an HDB_CONFIG template.
• utility/install/installer.ts: Stages the composed install config before mountHdb(), validates it via beforeCreateTables before creating system tables, and backs up and writes the staged config to HDB_CONFIG or harper-config.yaml; ⚠️ Look hardest: rollbackInstall() uses pre-install snapshots of root entries, storage directories, and candidate system database paths so rollback removes only artifacts created during the failed install and restores any pre-existing HDB_CONFIG.
• utility/environment/environmentManager.ts: Updates setProperty() so passing undefined deletes the key from appliedOverrides and clears the property instead of leaving a stale worker override.
• security/keys.ts: Updates the comment in updateConfigCert() to note that filterArgsAgainstRuntimeConfig() filters both HARPER_CONFIG and HARPER_SET_CONFIG keys.
• bin/run.ts and utility/mount_hdb.ts: Tracks installedDuringInit in bin/run.ts so first-time boot skips calling updateConfigValue(false) a second time after install() already persisted the config, and adds an optional beforeCreateTables callback in utility/mount_hdb.ts that runs after directory creation and before createTables().
• DESIGN.md, config/DESIGN.md, and resources/DESIGN.md: Documents the staged install configuration pipeline and stageInstallConfig() ordering in DESIGN.md, config/DESIGN.md, and resources/DESIGN.md.

🤔 How should empty maps in an HDB_CONFIG template interact with HARPER_DEFAULT_CONFIG?

An empty map where defaultConfig.yaml defines a non-empty section (such as http: {}) is backfilled from defaults. It stays overridable by HARPER_DEFAULT_CONFIG. Any other empty map (such as myComponent: {} or operationsApi.tls: {}) is recorded as 'user'-authored and preserved. Allowing HARPER_DEFAULT_CONFIG to populate all empty maps is a two-line change in collectLeafDotPaths().

✅ Verification

• Ran npx mocha unitTests/install/installer.test.js unitTests/config/configUtils.test.js unitTests/config/harperConfigEnvVars-config.test.js unitTests/config/harperConfigEnvVars-filter.test.js unitTests/utility/environment/environmentManager.test.js (180 passing), verifying real install() runs across RocksDB and LMDB, HDB_CONFIG templates, HARPER_DEFAULT_CONFIG / HARPER_SET_CONFIG / STORAGE_* layers, pre-table validation failures, and snapshot-scoped rollback.
• Ran npm run lint and npm run check:design-docs with zero warnings or errors.
• PR QA: Does a first-time server boot with HDB_CONFIG and HARPER_SET_CONFIG storage overrides start all worker threads on the same storage engine? — queued @ eb2bcdc, qa-pr-harper-3233-eb2bcdc

Added unit test coverage by file

• unitTests/install/installer.test.js: End-to-end install() tests across RocksDB and LMDB for every config env layer, HDB_CONFIG templates, boot-flow initialize(), pre-table validation failure, and snapshot-scoped rollback.
• unitTests/config/configUtils.test.js: Unit tests for stageInstallConfig(), validateStagedInstallConfig(), writeStagedInstallConfig(), HDB_CONFIG default backfilling, and empty-section source attribution (http: {}, myComponent: {}, operationsApi.tls: {}).
• unitTests/config/harperConfigEnvVars-config.test.js and unitTests/config/harperConfigEnvVars-filter.test.js: Tests for filterArgsAgainstRuntimeConfig() with HARPER_CONFIG and legacy aliases, prepareRuntimeEnvConfig() with userPaths, and ancestor/descendant source precedence across HARPER_DEFAULT_CONFIG, HDB_CONFIG, and HARPER_SET_CONFIG.
• unitTests/utility/environment/environmentManager.test.js: Tests that setProperty(key, undefined) clears the property and removes its entry from getConfigOverrides().

_{🤖 Generated by Google Gemini; posted via @kriszyp.}

Release target: main — 134 changed lines predate this shallow clone's history, so who introduced them is unknown
Related PRs: #3078 overlaps (both touch install() and rollbackInstall() in utility/install/installer.ts)
Complexity: complicated

Deferred findings

• 53553ea78c754ada minor data-integrity — utility/install/installer.ts — Rollback removes all of ~/.harperdb (pre-existing keys/stale props) whenever this run wrote the props file — filed Installer rollback removes entire ~/.harperdb directory instead of only newly created boot properties #3230
• 0374edb02dca6a99 minor documentation — config/harperConfigEnvVars.ts — HARPER_CONFIG now beats individual env vars for every key at install and runtime; breaking precedence change — filed docs(config): document HDB_CONFIG template backfill and HARPER_CONFIG precedence in external docs #3231
• 9b856519c7f02598 minor data-integrity — config/harperConfigEnvVars.ts — Env-owned storage.engine is deleted when the env var is removed; next boot uses the default engine against the existing system DB — filed stageRocksCompression leaves temporary storage compression overrides in envManager after install #3218
• 53cb954cf387c47c nit documentation — config/configUtils.ts — Template backfill pins every current default into the operator's HDB_CONFIG, rewritten in place — filed docs(config): document HDB_CONFIG template backfill and HARPER_CONFIG precedence in external docs #3231

_{Review-Coverage: authored=gemini; ran=muse,claude,cursor-composer,cursor-muse; adjudicated=domain; declined=cursor-grok,cursor-kimi; rounds=16; full=4 @ eb2bcdc}

_{Review-Attention: study ~14m (sensitive: keys.ts; decisions: do-less-alternative, hdb-config-in-place-rewrite, snapshot-scoped-rollback, empty-template-section-semantics, setproperty-undefined-clears) @ eb2bcdc}

[gemini-code-assist]

Code Review

This pull request introduces a staged configuration flow during installation, allowing the full install configuration to be composed, validated, and staged in-memory before the database is mounted and written to disk. It adds support for backfilling missing defaults from defaultConfig.yaml when a user-supplied HDB_CONFIG template is provided, while tracking user-authored paths to preserve the precedence ladder (HARPER_DEFAULT_CONFIG < user/HDB_CONFIG/CLI < HARPER_CONFIG < HARPER_SET_CONFIG). Additionally, it enhances argument filtering to include HARPER_CONFIG alongside HARPER_SET_CONFIG, improves the robustness of the installation rollback process to preserve pre-existing operator files and restore pre-seeded configurations, and updates setProperty to clear overrides when passed undefined. As there are no review comments, I have no feedback to provide.

[kriszyp]

Thanks for taking this on. The staged install config looks like the right shape. One acceptance case worth adding explicitly, since the PR text frames the bug as engine and compression and does not mention storage.path.

Case: first-boot storage.path outside ROOTPATH. On main, mountHdb runs before createConfigFile (utility/install/installer.ts), and both getBaseSchemaPath (initializePaths.js:22-32) and resolveDatabaseStorageRoot (resources/databases.ts:2647-2648) read getConfigPath(STORAGE_PATH), which is still undefined then. The install-time system tables land in <rootDir>/database/system while later config points the rest of Harper at the external path, so main/0 and the http workers disagree about where system lives.

Test shape, for both RocksDB and LMDB, with HARPER_SET_CONFIG='{"storage":{"path":"<extDir>"}}' (no STORAGE_PATH env var or flag) on a first boot:

• REST request with Basic auth as the install admin succeeds on an http worker (on main it returns 401 Table hdb_user not found).
• Operations describe_all on an http worker lists system, and <extDir> holds the system database, not only the component databases.
• Two-process variant (harper install, then harper run with the same config) starts without database 'system' does not exist.

By my read of stageInstallConfig() this is already covered, since getConfigPath(STORAGE_PATH) would see <extDir> before mountHdb. I have not run it against the PR head (eb2bcdc84), so a test would settle it.

Possible gap, for after merge. A pre-seeded <rootDir>/harper-config.yaml (documented as HDB_CONFIG=/existing/rootpath/harper-config.yaml harper) looks uncovered. It fails earlier, in the module-load envMgr.initSync() (bin/harper.ts into bin/cliOperations.ts:6): noBootFile() and isHdbInstalled() treat a root config as already installed and skip install, and the PR changes bin/run.ts only. I only confirmed those guards are unchanged on the PR; I have not reproduced it there. I will file it separately after this merges rather than widen the PR.

— Claude Sonnet 5.5