diff --git a/docs/plans/coverage-ingestion.md b/docs/plans/coverage-ingestion.md new file mode 100644 index 0000000..647f209 --- /dev/null +++ b/docs/plans/coverage-ingestion.md @@ -0,0 +1,169 @@ +# Static coverage ingestion (Istanbul JSON + LCOV → `coverage` table) + +> **Status:** in design (no code) · **Backlog:** [research/fallow.md § C.11](../research/fallow.md#tier-c--ship-eventually-months-high-payoff-large-surface). Delete this file when shipped (per [`docs/README.md` Rule 3](../README.md)). + +## Goal + +Ingest static coverage artifacts — Istanbul JSON (`coverage-final.json`) and LCOV (`lcov.info`) — into the codemap index so structural queries can compose coverage filters in pure SQL, without bolting Codemap to a runtime tracer or paid coverage stack. Both formats land in v1 so every coverage tool (vitest / jest / c8 / nyc / `bun test --coverage`) is a first-class consumer on day one. + +The killer recipe this unlocks: + +```sql +-- "What's structurally dead AND untested?" — single query, two evidence axes. +-- `calls` is name-keyed (no symbol-id FK, see `db.ts` `CallRow`), so the +-- "no callers" predicate is name-only / lossy across cross-file collisions +-- (acceptable v1 limitation — see D11). The `coverage` join uses the natural +-- key (file_path, name, line_start) so it survives `--full` reindex (D6). +SELECT s.name, s.file_path, c.coverage_pct +FROM symbols s +LEFT JOIN coverage c + ON c.file_path = s.file_path + AND c.name = s.name + AND c.line_start = s.line_start +WHERE s.is_exported = 1 + AND NOT EXISTS (SELECT 1 FROM calls WHERE callee_name = s.name) + AND COALESCE(c.coverage_pct, 0) = 0 +ORDER BY s.file_path, s.line_start; +``` + +Today an agent has to run two tools (`codemap` + a coverage reader) and join in JS. After this lands it's one `query` + one `JOIN`. + +## Why + +- **Codemap is structural-only today.** Every "is this dead?" query has a structural false-positive rate (the §0 fallow audit found an 8-file widget pack with non-zero structural fan-in but zero runtime usage). Coverage is the **complementary evidence axis** — `structural fan-in = 0` AND `runtime coverage = 0` is the high-confidence "dead" predicate. +- **Static ingestion is free.** Istanbul JSON (`vitest`, `jest`, `c8 --reporter=json`, `nyc`) and LCOV (`bun test --coverage`, `c8 --reporter=lcov`, every legacy stack) cover the entire test-runner ecosystem. We're not building a coverage tracer — we're reading the artifacts those tools already produce. +- **Fallow's runtime intelligence is paid.** Static coverage ingestion gets ~80% of the agent value (the "is X dead?" predicate above) without entering Fallow's V8/production-beacon territory (explicit non-goal per [research/fallow.md § D.16](../research/fallow.md#defer--skip)). +- **Composes with `codemap impact`.** `impact --direction up --depth 0` returns callers; joining `coverage` on the result tells the agent "this symbol has 12 callers but only 2 of them are hit by tests" — refactor risk in one query. + +## Agent journey (the "first-class" axis) + +Every common agent question this feature unlocks must be a one-verb call, not "compose this JOIN yourself" — that's what "fully capable, no half-way APIs" means in practice. The v1 surface: + +| Agent asks | v1 verb | Backed by | +| ----------------------------------------- | -------------------------------------------------------------- | ------------------------ | +| "Is `legacyClient` tested?" | `query "SELECT * FROM coverage WHERE name = '…'"` | `coverage` tbl | +| "What's structurally dead AND untested?" | `query --recipe untested-and-dead` | D13 | +| "Rank files by test coverage" | `query --recipe files-by-coverage` | D13 (covers D2 deferral) | +| "Worst-covered exported symbols (top 20)" | `query --recipe worst-covered-exports` | D13 | +| "Coverage of these specific symbols" | `query "SELECT … FROM coverage WHERE name IN (…)"` | `coverage` tbl | +| "Did coverage change since base?" | `--save-baseline` + `--baseline` (existing primitive composes) | B.6 | +| "Refactor risk: callers vs coverage" | `impact ` JSON piped into a `coverage` LEFT JOIN | impact + coverage | + +Recipes 2 and 3 are added because deferring the `file_coverage` rollup table (D2) would otherwise force the agent to compose `GROUP BY` queries by hand — half-baked surface. Bundling the recipes keeps the schema lean (D2) AND the agent surface complete (D13). + +## Sketched layout + +### Schema (D1, D6 natural-key fix) + +```sql +-- Single table; symbols-side denormalisation rejected (D1). Natural-key PK +-- (file_path, name, line_start) — NOT a FK to symbols.id — so rows survive +-- the symbols-table drop-and-recreate cycle on every `--full` reindex (D6). +-- file_coverage rollup deferred to v1.x (D2). +CREATE TABLE coverage ( + file_path TEXT NOT NULL, + name TEXT NOT NULL, + line_start INTEGER NOT NULL, + coverage_pct REAL, -- NULL when total_statements = 0 (D5 edge) + hit_statements INTEGER NOT NULL, + total_statements INTEGER NOT NULL, + PRIMARY KEY (file_path, name, line_start) +) STRICT, WITHOUT ROWID; + +-- Index that mirrors the typical join shape `symbols.{file_path,name,line_start}`. +CREATE INDEX idx_coverage_file_name ON coverage(file_path, name); + +-- Meta row: timestamp + source path of the last successful ingest. Lets agents +-- check freshness without a separate verb. NULL when no coverage ever ingested. +-- `source` lives here (single ingest at a time), not as a per-row column. +INSERT INTO meta (key, value) VALUES ('coverage_last_ingested_at', ''); +INSERT INTO meta (key, value) VALUES ('coverage_last_ingested_path', ''); +INSERT INTO meta (key, value) VALUES ('coverage_last_ingested_format', 'istanbul'); -- or 'lcov' +``` + +**Why no FK / CASCADE:** `symbols.id` is `INTEGER PRIMARY KEY AUTOINCREMENT`; on `--full` reindex `dropAll()` drops `symbols` and `createTables()` recreates it with fresh IDs. A FK with CASCADE would wipe every coverage row on every full rebuild even if `coverage` itself were excluded from `dropAll()` — see D6 for the full unwind. Natural key sidesteps it. Orphan rows (file deleted from project) get cleaned by an explicit one-statement sweep at the end of every ingest: + +```sql +DELETE FROM coverage WHERE file_path NOT IN (SELECT path FROM files); +``` + +### CLI + +```text +codemap ingest-coverage [--json] +``` + +- `` — required. One of: `coverage-final.json` (Istanbul), `lcov.info` (LCOV), or a directory — engine probes the directory for either filename, errors if both or neither are present (no precedence guessing — explicit is better than implicit). +- `--json` — emit `{ingested: {symbols: N, files: M}, skipped: {unmatched_files: K, statements_no_symbol: S}, pruned_orphans: O, format: "istanbul"|"lcov"}` envelope on stdout. + +**No `--source` flag** — format is auto-detected from extension (`.json` → istanbul, `.info` → lcov). Adding a flag for a format the engine can detect is API noise; if a user ever has a misnamed file, they can rename it (one-liner) cheaper than codemap can grow a flag for it. + +**No `--prune` flag** — orphan cleanup is unconditional after every ingest (one DELETE; cheap; the only time coverage rows for a deleted file would be valid evidence is "this file used to exist" which is git's job, not codemap's). + +Decoupled from `codemap` (the index command) on purpose (D4) — coverage runs once per test invocation, not once per file edit. + +## Decisions + +| # | Decision | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| D1 | **Separate `coverage` table, not columns on `symbols`.** Resolves the open question in [research/fallow.md § 6](../research/fallow.md#6-open-questions). Three reasons: (a) coverage shape evolves independently of structural columns (per-branch, per-function, per-line metrics layered later) — denormalising churns `SCHEMA_VERSION` every time; (b) `symbols` rows exist for every TS / TSX file; coverage rows only for tested files — joining via `LEFT JOIN coverage` keeps NULL semantics explicit; (c) lifecycle independence — `symbols` re-creates on every `--full` reindex with fresh auto-increment IDs; the natural-key PK in `coverage` (D6) survives that churn without re-ingest. | +| D2 | **One table in v1; defer `file_coverage` rollup to v1.x — but ship the agent-facing answer as a bundled recipe.** `SELECT file_path, SUM(hit_statements), SUM(total_statements) FROM coverage GROUP BY file_path` answers "files ranked by coverage" with a single index-scan on `idx_coverage_file_name`. A separate rollup table doubles the source-of-truth surface and adds an UPSERT path; promote only after a real consumer query proves the GROUP BY is the bottleneck. The agent surface stays first-class via the bundled `files-by-coverage` recipe (D13) — no agent ever has to compose the GROUP BY by hand. Same "don't pre-emptively widen" discipline as D5; same "first-class via recipe" pattern as D13. | +| D3 | **Both Istanbul JSON and LCOV in v1; c8 raw V8 traces never.** Earlier draft deferred LCOV to v1.x; reversed because shipping a coverage feature that excludes `bun test --coverage` users (which ships LCOV / text reporters only — verified via `bun test --help`) is half-baked for codemap's own primary runtime, and the "no half-way APIs" principle bans it. Istanbul JSON (`coverage-final.json`) covers `c8`, `nyc`, `vitest --coverage --coverage.reporter=json`, `jest --coverage --coverageReporters=json`. LCOV (`lcov.info`) covers `bun test --coverage`, `c8 --reporter=lcov`, every legacy stack. Together: every modern coverage tool emits at least one of these — no consumer left waiting. Two parser front-ends share one `upsertCoverageRows()` core, so the second format costs ~1 LoC per parser plus the regex tokenizer for LCOV. Raw V8 traces (`*.cpu.profile` / `coverage-c8/*.json`) stay out of scope — Fallow's paid moat, requires a runtime tracer Codemap doesn't ship. | +| D4 | **One-shot `codemap ingest-coverage `, NOT auto-detected during `codemap` runs.** Reasons: (a) `codemap` is sub-100ms cold-start; auto-probing for `coverage/coverage-final.json` adds a `stat` and grows the surface for "where did codemap look for coverage?"; (b) coverage cadence (once per test run) is decoupled from index cadence (every file edit) — coupling them means stale coverage on every save; (c) explicit verb makes the agent's mental model trivial: "tests ran → `codemap ingest-coverage` → `codemap query`". `codemap watch` does NOT auto-ingest coverage on `coverage-final.json` change (separate concern; revisit if demand materialises). | +| D5 | **Statement coverage only in v1.** Istanbul tracks statement / branch / function / line coverage separately. `coverage_pct` is **statement** coverage (the most stable signal across runners — function coverage misses anonymous closures, branch coverage explodes for switch / ternary). v1.x can add `branch_coverage_pct` / `function_coverage_pct` columns once a real consumer asks. Don't pre-emptively widen. | +| D6 | **Natural-key PK `(file_path, name, line_start)` — no FK to `symbols.id`.** Earlier draft used `symbol_id REFERENCES symbols(id) ON DELETE CASCADE`; CodeRabbit-style self-audit caught that this would wipe every coverage row on every `--full` reindex (because `dropAll()` drops `symbols` → CASCADE fires → recreated `symbols` get fresh auto-increment IDs that don't match the old ones anyway). Natural key sidesteps the entire CASCADE hazard. Trade-off: orphan rows when a file is deleted from the project; cleaned by a single `DELETE FROM coverage WHERE file_path NOT IN (SELECT path FROM files)` at the end of every ingest. `coverage` is excluded from `dropAll()` (joins the `query_baselines` precedent) so the natural-key rows persist across `--full`. Re-ingest is the user's explicit refresh verb only when test results actually changed. | +| D7 | **Symbol mapping: innermost-wins line-range projection.** Istanbul's `statementMap` is line-keyed; we project per-statement hits onto the **innermost** enclosing symbol — `line_start ≤ stmt_line ≤ line_end` ordered by `(line_end - line_start) ASC LIMIT 1`. Innermost-wins is required because symbols nest (class methods inside classes, closures inside functions); attributing one statement to all enclosing scopes would inflate `total_statements` 2-3× on real codebases. Symbols whose range covers 0 statements (interfaces, type aliases) get no coverage row → `NULL` in `LEFT JOIN coverage`. (No prior table uses this projection — `markers` are line-pinned, `calls` are name-keyed; the projection is novel for this plan.) Statements that fall outside every symbol range (top-level expression statements, side-effect imports) increment `skipped.statements_no_symbol` for observability — no row written. | +| D8 | **Path normalisation: project-relative, forward-slashed.** Istanbul writes absolute paths; we strip `/` and replace `\\` with `/` to match `files.path`. Files outside the project root land in `skipped.unmatched_files`. Same projection `toProjectRelative()` (in `validate-engine`) already does — reuse the helper instead of rewriting it. | +| D9 | **MCP / HTTP exposure: column in `query` results, NOT a separate `coverage` tool.** The killer recipe (top of this doc) is one SQL query — `query` / `query_recipe` already returns `coverage_pct` as a column when the SELECT asks for it. A standalone `coverage` MCP tool would duplicate the surface; revisit only if a consumer ships a wrapper script that proves the SQL ergonomic gap is real. | +| D10 | **`codemap audit` integration deferred.** Adding `--delta coverage` to `audit` is the natural next step (flag "files where `coverage_pct` dropped >5% vs `--base`") but layered on top of D1–D9. Track as v1.x backlog; ship the ingester + raw schema first. | +| D11 | **Ingester lives in `application/coverage-engine.ts`.** Same engine-vs-CLI split as `impact` / `audit`: pure ingester (`ingestIstanbul` / `ingestLcov`, both calling `upsertCoverageRows`) consumed by `cli/cmd-ingest-coverage.ts`. No MCP / HTTP transport in v1 (D9). Engine is unit-testable against fixture artifacts without spinning up the CLI. Edge cases the engine guards: `total_statements = 0 → coverage_pct = NULL` (not 0; "untested" and "no testable code" are different); name-collision tolerance: same `(file_path, name, line_start)` is unique by construction (two functions can't start on the same line), so the natural-key PK never collides — but cross-file name collisions (`init` in `a.ts` and `b.ts`) mean the killer recipe's `callee_name = s.name` predicate is name-only / lossy in v1. **Mitigation shipped in v1**: the bundled `untested-and-dead.md` documents the limitation and shows three concrete narrowing patterns the agent can apply (scope by `file_path LIKE 'src/api/%'`, exclude framework re-exports via `is_default_export = 0`, restrict to `is_exported = 1` already in the recipe). v1.x can add a `caller_file_path` column to `calls` for fully-precise resolution if a consumer's narrowing pattern proves insufficient. | +| D13 | **Bundled recipe shelf — every common agent question gets a `--recipe` verb, not "write your own SQL".** Three v1 recipes in `templates/recipes/` cover the agent journey end-to-end: (1) `untested-and-dead.{sql,md}` — exported symbols with no callers and zero coverage (the killer recipe); (2) `files-by-coverage.{sql,md}` — files ranked ascending by coverage_pct, surfacing the GROUP BY view that makes D2's deferral non-blocking; (3) `worst-covered-exports.{sql,md}` — exported symbols sorted by ascending coverage_pct, with a configurable `LIMIT`-via-frontmatter for the "show me the top 20" agent ask. Each recipe `.md` carries an `actions` block (per [PR #26](https://github.com/stainless-code/codemap/pull/26)) so agents see the suggested follow-up per row (e.g. "open-deprecation-issue", "add-test-suite"). All three appear in `--recipes-json` automatically — agents discover them at session start without reading docs. | +| D12 | **Schema bump = minor changeset.** Adds one table (`coverage`) + three meta keys; doesn't break any existing query. Per [`.agents/lessons.md`](../../.agents/lessons.md) "changesets bump policy" (verbatim: _reserve minor for schema-breaking changes that force a `.codemap.db` rebuild — matches 0.2.0 precedent: new tables/columns/`SCHEMA_VERSION` bump_), new tables + `SCHEMA_VERSION` bump = minor. The bump triggers `dropAll()` on next `codemap` run; the `coverage` table is absent on existing installs until first ingest (no migration needed — empty is the correct initial state). Subsequent `SCHEMA_VERSION` bumps preserve coverage data via the `dropAll()` exclusion (D6). | + +## Tracer-bullet plan + +Per [`tracer-bullets.mdc`](../../.cursor/rules/tracer-bullets.mdc) — vertical slices, each shippable on its own. + +| # | Tracer | Acceptance | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | **Schema + bump** in `src/db.ts`. Add the `coverage` table DDL (natural-key PK per D6 — no FK to `symbols.id`); bump `SCHEMA_VERSION`; add `coverage` to the `query_baselines`-style "preserve across `--full`" exclusion in `dropAll()`. Add `idx_coverage_file_name`. New `meta` keys (`coverage_last_ingested_at` / `_path` / `_format`) need no DDL — `meta` is `(key, value)` already. Unit test (`src/db.test.ts`) inserts + reads back one row; verifies the table survives a `dropAll()` + `createSchema()` round-trip; verifies the orphan-cleanup DELETE removes rows whose `file_path` is no longer in `files`. | Table exists; survives `--full`; orphan sweep works. | +| 2a | **Shared engine core** in `src/application/coverage-engine.ts` (new). `upsertCoverageRows({db, rows, projectRoot}) → {ingested, skipped, pruned_orphans}` — format-agnostic: takes a normalised `CoverageRow[]` (`{file_path, line, hit_count}`-shape), maps each row to the **innermost** enclosing symbol via the SQL `(line_end - line_start) ASC LIMIT 1` projection (D7), upserts `coverage` keyed on `(file_path, name, line_start)`, writes the meta keys, runs the orphan-cleanup DELETE. Computes `coverage_pct = total_statements > 0 ? hit / total * 100 : NULL`. Reuses `toProjectRelative` from `validate-engine` (D8). Plus `ingestIstanbul({db, payload, projectRoot})` — parses the Istanbul shape (`{ [absPath]: {statementMap, s, fnMap, f, branchMap, b, path} }`), normalises to `CoverageRow[]`, calls `upsertCoverageRows`. Unit tests cover both the shared core (fresh ingest, re-ingest UPSERT idempotence, unmatched file, statement-outside-symbol, nested-closure-innermost-wins, `total_statements = 0 → NULL`, orphan cleanup) and the Istanbul parser (statement-map shape, absolute-path normalisation, Windows-path handling). | Both pieces pure; deterministic upsert; Istanbul parser tested in isolation against fixture JSON. | +| 2b | **LCOV parser** in `src/application/coverage-engine.ts` — `ingestLcov({db, payload, projectRoot})`. Pure regex tokenizer over the LCOV record format (`SF:` / `DA:,` / `end_of_record`); normalises to the same `CoverageRow[]`, calls `upsertCoverageRows` — zero new write-side code. Unit tests cover: well-formed LCOV with multiple `SF` records, `DA` lines with hit / miss / both, malformed records (missing `end_of_record` → error), unmatched file. Real `bun test --coverage` output captured into a fixture for round-trip testing. | Pure parser; passes round-trip on a real `bun test --coverage` artifact. | +| 3 | **CLI verb** `cli/cmd-ingest-coverage.ts` — parses `` + `--json`. Auto-detects format: file ending `.json` → istanbul, `.info` → lcov, directory → probes for `coverage-final.json` / `lcov.info` (errors if both or neither). Reads JSON via the established runtime split (`Bun.file(path).json()` on Bun for the native-parser perf win; `readFile + JSON.parse` on Node — see [`packaging.md § Node vs Bun`](../packaging.md#node-vs-bun), mirrors `config.ts`); reads LCOV via `Bun.file(path).text()` / `readFile`. Dispatches to `ingestIstanbul` or `ingestLcov`. `main.ts` dispatcher gains the verb between existing entries. Help text in `bootstrap.ts` lists the new command. Unit tests cover: file-not-found, malformed JSON / LCOV, ambiguous directory (both files), empty directory, `--json` envelope shape. | `bun src/index.ts ingest-coverage ` and `` both write rows; `--help` lists the verb. | +| 4 | **Fixture coverage data + bundled recipe shelf (D13)** — ship two fixtures: `fixtures/minimal/coverage/coverage-final.json` (Istanbul) and `fixtures/minimal/coverage/lcov.info` (LCOV) covering the same partial coverage shape (e.g. `usePermissions` 100%, `legacyClient` 0%, `now` 50%). Three bundled recipes in `templates/recipes/` per D13: `untested-and-dead.{sql,md}` (killer recipe, with the three name-collision narrowing patterns from D11 in the `.md`), `files-by-coverage.{sql,md}` (replaces the deferred `file_coverage` table per D2), `worst-covered-exports.{sql,md}` (top-N exported symbols by ascending coverage_pct). Each `.md` includes a frontmatter `actions` block for per-row agent hints. Five golden recipes under `fixtures/golden/minimal/`: `coverage-istanbul.json`, `coverage-lcov.json` (cross-format equivalence), `untested-and-dead.json`, `files-by-coverage.json`, `worst-covered-exports.json`. Adds fixtures + all three recipes to `fixtures/minimal/README.md` "What's exercised" table. The pre-enriched `legacyClient` / `now` / `epochMs` deprecated symbols (PR #55) directly test the join axis. | Both ingesters produce identical golden rows; all three recipes queryable via `--recipe`; `--recipes-json` lists all three. | +| 5 | **Doc + agent rule + skill + changeset + plan deletion** — `docs/architecture.md` § Persistence wiring (new table, ingester engine, CLI verb, both formats, three bundled recipes), `docs/glossary.md` (`coverage_pct`, "Istanbul JSON", "LCOV", "static coverage ingestion", `untested-and-dead`, `files-by-coverage`, `worst-covered-exports`), `.agents/rules/codemap.md` + `templates/agents/rules/codemap.md` (Rule 10 lockstep — three new trigger-pattern rows: "What's untested?" → `coverage` table, "What's untested AND structurally dead?" → `--recipe untested-and-dead`, "Rank files by coverage" → `--recipe files-by-coverage`; also a `coverage` row in the table-shape table), skill `SKILL.md` `coverage` table row + recipes section update, README.md "What's exercised" + Use section (`bun test --coverage` LCOV worked example + `vitest --coverage` Istanbul worked example). Minor changeset (per D12). Plan deleted per `docs/README.md` Rule 3. | All docs consistent; agents see the `coverage` table + all three bundled recipes + new trigger patterns on next `codemap agents init`. | + +## Performance considerations + +- **Ingest cost** — one parse pass (Istanbul JSON or LCOV) + linear scan of statements + one symbol-projection SQL per statement. Per-statement projection is the hot path; backed by `idx_symbols_file` it's an index-bounded lookup. On Bun, JSON parsing uses `Bun.file(path).json()` (native parser, materially faster than V8 `JSON.parse` on multi-MB Istanbul payloads); on Node it's `readFile + JSON.parse`. LCOV is line-by-line text on both runtimes. No published benchmark yet — measurement target during tracer 2. +- **Read cost** — `coverage` is `WITHOUT ROWID` on `(file_path, name, line_start)`. The killer recipe's `LEFT JOIN coverage ON file_path AND name AND line_start` is a 3-column composite-PK lookup per row; `idx_coverage_file_name` covers the prefix. +- **Storage** — three INTEGER + one REAL + two short TEXT per row; small. Real disk impact dominated by symbol count, not coverage shape. +- **No background worker.** Ingest is single-pass; reindex is unaffected. + +## Alternatives considered + +| Candidate | Why not | +| -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Add `coverage_pct REAL` + `is_runtime_hot INTEGER` to `symbols` directly** | Per D1: shape-coupling, NULL ambiguity, schema-bump amplification. The original sketch ([research/fallow.md § C.11](../research/fallow.md#tier-c--ship-eventually-months-high-payoff-large-surface)) listed this — D1 supersedes. | +| **Auto-detect `coverage/coverage-final.json` during `codemap` runs** | Per D4: cadence mismatch + auto-probe surface. The auto-detect path is also harder to reason about ("why does this index include coverage on machine A but not B?"); explicit verb is unambiguous. | +| **Defer LCOV to v1.x (Istanbul-only v1)** | Per D3: ships a coverage feature that excludes `bun test --coverage` users — half-baked for codemap's own primary runtime, banned by the "no half-way APIs" principle. Two parser front-ends + one shared upsert core costs ~one tracer's worth of code; not worth deferring. | +| **Keep `--source istanbul\|lcov` flag** | API noise: format is auto-detectable from extension (`.json` / `.info`); a flag for "tell codemap what it can already see" is over-engineered. Misnamed files can be renamed (one-liner) cheaper than codemap can grow a flag. | +| **Embed runtime coverage tracer (V8 / Istanbul beacons)** | Out of scope per [research/fallow.md § D.16](../research/fallow.md#defer--skip) and [roadmap.md § Non-goals](../roadmap.md#non-goals-v1). This plan reads artifacts; it does not produce them. | +| **`coverage` rows keyed on `symbol_id` with `ON DELETE CASCADE`** | Original draft. Rejected per D6: `dropAll()` drops `symbols` on every `--full`, CASCADE wipes coverage, recreated `symbols` get fresh auto-increment IDs anyway → coverage permanently lost without re-ingest. The natural-key approach (D6) sidesteps the entire FK/CASCADE hazard at the cost of one explicit orphan-cleanup DELETE per ingest. | +| **Separate `file_coverage` rollup table in v1** | Per D2: aggregateable in one indexed `GROUP BY` on the symbol-level table; doubling the source-of-truth surface without a benchmark is premature. Promote in v1.x once a real query proves the cost. | +| **Separate MCP `coverage` tool returning `{symbol, pct, hit, total}` envelopes** | Per D9: the column-in-`query`-results path is composable with every existing recipe + ad-hoc SQL; a standalone tool would force a parallel surface for one column. Revisit if SQL composition proves too verbose for agents. | +| **Inline coverage into `codemap audit` v1 (`--delta coverage`)** | Per D10: the plan stays small. Audit-side delta is a clean follow-up once the raw schema lands. | +| **Persist coverage in a sibling JSON file (`/coverage.json`)** | Forces every consumer to re-implement the join; loses SQL composability. The whole point of Codemap is "it's a SQL index" — keep coverage in the same DB. | + +## Out of scope + +- **Branch / function / line coverage breakdowns** — D5; v1.x once a consumer asks with a concrete query. +- **`file_coverage` rollup table** — D2; v1.x with a benchmark. +- **Cross-file callee disambiguation** — D11; the killer recipe's `callee_name = s.name` is name-only / lossy in v1. v1.x can add a `caller_file_path` column to `calls` if a real consumer needs the precision. +- **Coverage diff against baseline** — `--save-baseline` / `--baseline` already covers arbitrary query result snapshots; coverage queries inherit it for free. A first-class `coverage_diff` would duplicate. +- **Coverage trend over time** — adjacent to telemetry; not in v1. Consumers can `--save-baseline coverage-snapshot-` periodically. +- **CI verdict / threshold logic** — same condition as `codemap audit verdict`: defer until a consumer ships a `jq` script that proves the threshold shape. +- **`--delta coverage` in `codemap audit`** — D10; layered on top of the v1 schema. +- **Auto-running tests** — Codemap reads coverage artifacts; it doesn't invoke any test runner. Test orchestration is the user's CI. +- **Source-map-aware coverage** — Istanbul's `statementMap` is post-transform; coverage rows reflect the compiled file's structure. Source-map walking to original `.ts` lines is deferred (most TS runners — `vitest`, `jest` — instrument the pre-compile source already). +- **`codemap watch` auto-reingest** — D4 explicitly excludes; revisit only if a consumer writes a watcher that re-runs `bun test --coverage` automatically. diff --git a/docs/roadmap.md b/docs/roadmap.md index d6ebfd1..9ad5a8e 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -35,6 +35,7 @@ Codemap stays a structural-index primitive that other tools can consume. Out of ## Backlog +- [ ] **Static coverage ingestion** — `codemap ingest-coverage ` reads Istanbul `coverage-final.json` into a `coverage` table, joinable to `symbols` for "what's structurally dead AND untested?" queries. Plan: [plans/coverage-ingestion.md](./plans/coverage-ingestion.md). Adapted from [research/fallow.md § C.11](./research/fallow.md#tier-c--ship-eventually-months-high-payoff-large-surface). - [ ] **`codemap audit` verdict + thresholds** (v1.x) — `verdict: "pass" | "warn" | "fail"` driven by `codemap.config.audit.deltas[].{added_max, action}`. Triggers: two consumers ship `jq`-based threshold scripts with similar shapes, OR one consumer asks with a concrete config sketch. Until then, raw deltas + consumer-side `jq` is the CI exit-code idiom. - [ ] **Monorepo / workspace awareness** — discover workspaces from `pnpm-workspace.yaml` / `package.json` and index per-workspace dependency graphs - [ ] **Cross-agent handoff artifact** — _speculative_; layered prefix/delta JSON written on session-stop, read on session-start. Complementary to indexing rather than core to it; revisit if user demand emerges