Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,17 +57,18 @@ claude mcp add -s user -t stdio codebase-intelligence -- npx -y codebase-intelli

## Features

- **19 CLI commands** for architecture analysis, dependency impact, duplicate families, improvement opportunities, dead code detection, search, CI rules, and agent setup
- **20 CLI commands** for architecture analysis, dependency impact, duplicate families, Highways convergence, improvement opportunities, dead code detection, search, CI rules, and agent setup
- **Machine-readable JSON output** (`--json`) for automation and CI pipelines
- **Auto-cached index** in `.codebase-intelligence/` for fast repeat queries
- **Cache migration facts** in JSON (`cacheDir`, `legacyCacheDir`, `migrated`, `gitignoreUpdated`, `warnings[]`)
- **11 architectural metrics** — PageRank, betweenness, coupling, cohesion, tension, churn, complexity, blast radius, dead exports, test coverage, escape velocity
- **Symbol-level analysis** — callers/callees, symbol importance, impact blast radius
- **BM25 search** — ranked keyword search across files, symbols, and type/shape facts
- **Process tracing** — detect entry points and execution flows through the call graph
- **Highways analysis** — find repeated routes that bypass canonical dataflow paths
- **Community detection** — Louvain clustering for natural file groupings
- **Agent adoption** — `init` writes per-agent instruction files + installs a skill so AI agents query CI before grep/read
- **MCP parity (secondary)** — same analysis and rules gate available as 18 MCP tools, 2 prompts, and 3 resources
- **MCP parity (secondary)** — same analysis and rules gate available as 19 MCP tools, 2 prompts, and 3 resources

## Installation

Expand Down Expand Up @@ -110,6 +111,7 @@ codebase-intelligence <command> <path> [options]
| `impact` | Symbol-level blast radius |
| `rename` | Reference discovery for rename planning |
| `processes` | Entry-point execution flow tracing |
| `highways` | Repeated route convergence and canonical path opportunities |
| `clusters` | Community-detected file clusters |
| `check` | Rules-engine gate for CI, including opt-in dead-code and dependency gates |
| `init` | Set up AI agents to use CI — writes per-agent instruction files (skill opt-in via `--skill`) |
Expand All @@ -127,6 +129,8 @@ codebase-intelligence <command> <path> [options]
| `--min-tokens <n>` | Minimum duplicate token size for `duplicates` |
| `--skip-local` | Ignore duplicate families confined to one file |
| `--trace <id>` | Return token evidence for one duplicate family |
| `--operation <verb>` | Focus `highways` on one operation verb |
| `--shape <name>` | Focus `highways` on one type/DTO shape |

The scanner always excludes common generated and agent-workspace directories such as `.codebase-intelligence/`, legacy `.code-visualizer/`, `.next/`, `dist/`, `coverage/`, `.worktrees/`, and `.claude/worktrees/`.

Expand Down
8 changes: 5 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Core (shared computation)
| typed descriptors, input schemas, CLI/MCP adapters, result wrappers, text formatters
v
MCP (stdio) CLI (terminal/CI)
| 18 tools, 2 prompts, | 19 commands with text + JSON
| 19 tools, 2 prompts, | 20 commands with text + JSON
| 3 resources for LLMs | output for humans and CI
```

Expand All @@ -47,7 +47,8 @@ src/
duplication/index.ts <- Duplicate family detection + trace evidence
config/index.ts <- Config discovery + zod validation
rules/index.ts <- Rules engine + registry (check command + MCP check tool)
mcp/index.ts <- 18 MCP tools for LLM integration
highways/index.ts <- Repeated route convergence + bypass/cowpath evidence
mcp/index.ts <- 19 MCP tools for LLM integration
mcp/hints.ts <- Operation-keyed next-step hints for MCP tool responses
impact/index.ts <- Symbol-level impact analysis + rename planning
search/index.ts <- BM25 search engine
Expand Down Expand Up @@ -82,7 +83,7 @@ analyzeGraph(builtGraph, parsedFiles)
}

startMcpServer(codebaseGraph)
-> stdio MCP server with 18 tools, 2 prompts, 3 resources
-> stdio MCP server with 19 tools, 2 prompts, 3 resources

runOperation(operation, codebaseGraph, input, context)
-> { ok: true, data } | { ok: false, error, data? }
Expand All @@ -96,6 +97,7 @@ runOperation(operation, codebaseGraph, input, context)
- **Duplication families**: Parser stores deterministic function-body token streams on symbols. `duplicates` / `find_duplicates` groups symbols into strict, renamed, and near-miss clone families, with optional trace evidence for AI agents before refactors.
- **Dead-code gates**: `check` can opt into unused-file, unused-type, unused-member, and dependency hygiene rules. These rules use graph metrics plus local TypeScript AST facts and emit confidence/evidence in JSON and SARIF; dependency hygiene is scoped to the nearest package/workspace manifest.
- **Suppression hygiene**: `check` reports active and stale `ci-ignore-*` / `@expected-unused` suppressions, emits stale suppression findings via `no-stale-suppressions`, and treats `@public` exported type declarations as intentional public API while `@internal` stays checkable.
- **Highways H1**: `highways` / `analyze_highways` enumerates entry-to-sink call routes, groups repeated routes by sink, detects bypasses and cowpaths around an existing canonical node, and emits route chains, evidence, blast radius, recommendations, and a context pack for agents.
- **Shared graph-load pipeline**: CLI commands and MCP stdio startup both use `src/graph-loader/` for path checks, legacy cache migration, cache reuse, parse/build/analyze, optional persistence, and stderr progress events.
- **graphology**: In-memory graph with O(1) neighbor lookup. PageRank and betweenness computed via graphology-metrics.
- **Batch git churn**: Single `git log --all --name-only` call, parsed for all files. Avoids O(n) subprocess spawning.
Expand Down
18 changes: 16 additions & 2 deletions docs/cli-reference.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CLI Reference

19 commands for terminal and CI use. The 17 analysis commands have full parity with MCP tools and auto-cache the index to `.codebase-intelligence/`; `check` runs the rules gate; `init` sets up agent adoption.
20 commands for terminal and CI use. The 18 analysis commands have full parity with MCP tools and auto-cache the index to `.codebase-intelligence/`; `check` runs the rules gate; `init` sets up agent adoption.

## Commands

Expand Down Expand Up @@ -168,6 +168,16 @@ codebase-intelligence processes <path> [--entry <name>] [--limit <n>] [--json] [

**Output:** processes with entry point, steps, depth, modules touched.

### highways

Find repeated routes that should converge on one canonical operation path.

```bash
codebase-intelligence highways <path> [--operation <verb>] [--shape <name>] [--min-routes <n>] [--propose] [--trace <id>] [--json] [--force]
```

**Output:** route opportunities with `bypass` / `cowpath` kind, operation, shape, sink, canonical node, route chains, bypass routes, duplicated callees when present, evidence, blast radius, recommendation, and context pack.

### clusters

Community-detected file clusters (Louvain algorithm).
Expand Down Expand Up @@ -226,14 +236,18 @@ codebase-intelligence init [path] [--agents <list>] [--all] [--skill] [--gitigno
| `--mode <m>` | duplicates | Clone mode: strict, mild, weak |
| `--min-tokens <n>` | duplicates | Minimum function body token count (default: 30) |
| `--skip-local` | duplicates | Ignore families confined to one file |
| `--trace <id>` | duplicates | Return token evidence for one family id |
| `--trace <id>` | duplicates, highways | Return evidence for one family/opportunity id |
| `--scope <s>` | changes | Git diff scope: staged, unstaged, all |
| `--depth <n>` | dependents | Max traversal depth (default: 2) |
| `--cohesion <n>` | forces | Min cohesion threshold (default: 0.6) |
| `--tension <n>` | forces | Min tension threshold (default: 0.3) |
| `--escape <n>` | forces | Min escape velocity threshold (default: 0.5) |
| `--module <m>` | dead-exports | Filter by module path |
| `--entry <name>` | processes | Filter by entry point name |
| `--operation <verb>` | highways | Focus on one operation verb |
| `--shape <name>` | highways | Focus on one type/DTO shape |
| `--min-routes <n>` | highways | Minimum routes reaching a sink before reporting |
| `--propose` | highways | Include reroute proposal metadata |
| `--min-files <n>` | clusters | Min files per cluster |
| `--no-dry-run` | rename | Actually perform the rename (default: dry run) |
| `--format <fmt>` | check | Output format: text, json, sarif |
Expand Down
64 changes: 63 additions & 1 deletion docs/data-model.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Data Model

All types defined in `src/types/index.ts`. This is the single source of truth.
Core graph types are defined in `src/types/index.ts`. Feature result envelopes live beside their analyzer modules.

## Parser Output

Expand Down Expand Up @@ -171,6 +171,68 @@ CacheFacts {
}
```

## Highways Result

`highways --json` and MCP `analyze_highways` return deterministic route-convergence opportunities.

```typescript
HighwaysResult {
totalRoutes: number
totalSinks: number
totalOpportunities: number
operation?: string
shape?: string
minRoutes: number
opportunities: HighwayOpportunity[]
trace?: {
id: string
found: boolean
opportunity?: HighwayOpportunity
}
summary: string
}

HighwayOpportunity {
id: string // hwy-<kind>-<stable hash>
kind: "bypass" | "cowpath"
operation: string
shape?: string
sink: HighwayStep
canonicalNode: HighwayStep
routes: HighwayRoute[] // all routes reaching the sink
bypassRoutes: HighwayRoute[] // routes missing canonicalNode
duplicatedCallees?: HighwayStep[] // cowpath overlap with canonical node callees
evidence: string[]
blastRadius: number
recommendation: string
contextPack: {
summary: string
affectedRoutes: string[]
evidence: string[]
blastRadius: number
proposedCanonicalNode: HighwayStep
nextSafeCommand: string
}
}

HighwayRoute {
id: string
operation: string
shape?: string
entryPoint: HighwayStep
sink: HighwayStep
steps: HighwayStep[]
includesCanonical: boolean
confidence: "type-resolved" | "text-inferred"
}

HighwayStep {
id: string
file: string
symbol: string
}
```

## Check Findings

`check --json` and MCP `check` return deterministic findings plus a suppression
Expand Down
17 changes: 14 additions & 3 deletions docs/mcp-tools.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# MCP Tools Reference

18 tools available via MCP stdio.
19 tools available via MCP stdio.

Operation tools return JSON text payloads. Invalid operation inputs return `isError: true` with `{ "error": "..." }` using the same descriptor validation messages as CLI bad-argument exits.

Expand Down Expand Up @@ -169,7 +169,17 @@ Trace execution flows from entry points through the call graph.
**Use when:** "How does this app start?" "Trace request flow." "What are the entry points?"
**Not for:** Static file dependencies (use get_dependents).

## 17. get_clusters
## 17. analyze_highways

Detect repeated entry-to-sink routes that should converge on one canonical operation path.

**Input:** `{ operation?: string, shape?: string, minRoutes?: number, propose?: boolean, trace?: string }`
**Returns:** totalRoutes, totalSinks, totalOpportunities, opportunities[] (id, kind, operation, shape, sink, canonicalNode, routes[], bypassRoutes[], duplicatedCallees?, evidence[], blastRadius, recommendation, contextPack), optional trace

**Use when:** Enforcing dataflow discipline, finding ad-hoc routes, or planning canonical shared paths.
**Not for:** Raw execution flow listing (use get_processes).

## 18. get_clusters

Community-detected clusters of related files.

Expand All @@ -179,7 +189,7 @@ Community-detected clusters of related files.
**Use when:** "What files are related?" "Find natural groupings." Discovering emergent groupings that differ from directory structure.
**Not for:** Directory-based modules (use get_module_structure).

## 18. check
## 19. check

Run the configurable rules engine and gate on findings.

Expand Down Expand Up @@ -227,6 +237,7 @@ Rules: `no-comments` (off by default), `no-circular-deps` (error), `no-dead-expo
| "What changed?" | `detect_changes` |
| "Find all references to X" | `rename_symbol` |
| "How does data flow through the app?" | `get_processes` |
| "Which routes bypass canonical dataflow?" | `analyze_highways` |
| "What files naturally belong together?" | `get_clusters` |
| "What are the main areas?" | `get_groups` |
| "What rule violations exist? Lint this." | `check` |
27 changes: 19 additions & 8 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ Core (shared computation)
| typed descriptors, input schemas, CLI/MCP adapters, result wrappers, text formatters
v
MCP (stdio) + CLI
| MCP: 18 tools, 2 prompts, 3 resources for LLM agents
| CLI: 19 commands with formatted + JSON output for humans/CI
| MCP: 19 tools, 2 prompts, 3 resources for LLM agents
| CLI: 20 commands with formatted + JSON output for humans/CI
```

## Module Map
Expand All @@ -51,7 +51,8 @@ src/
operations/formatters.ts <- Result-object text formatters for CLI commands
parser/duplication.ts <- Function-body clone token extraction
duplication/index.ts <- Duplicate family detection + trace evidence
mcp/index.ts <- 18 MCP tools for LLM integration
highways/index.ts <- Repeated route convergence + bypass/cowpath evidence
mcp/index.ts <- 19 MCP tools for LLM integration
mcp/hints.ts <- Operation-keyed next-step hints for MCP tool responses
impact/index.ts <- Symbol-level impact analysis + rename planning
search/index.ts <- BM25 search engine
Expand Down Expand Up @@ -81,7 +82,7 @@ analyzeGraph(builtGraph, parsedFiles)
-> CodebaseGraph {
nodes, edges, symbolNodes, callEdges, symbolMetrics,
fileMetrics, moduleMetrics, forceAnalysis, stats,
groups, processes, clusters
groups, processes, highways, clusters
}

runOperation(operation, codebaseGraph, input, context)
Expand Down Expand Up @@ -261,7 +262,7 @@ The most dangerous files have: high churn + high coupling + low coverage.

# MCP Tools Reference

18 tools available via MCP stdio.
19 tools available via MCP stdio.

## 1. codebase_overview
High-level summary. Input: `{ depth?: number }`. Returns: totalFiles, totalFunctions, modules, topDependedFiles, metrics, and analysis mode/call graph precision.
Expand Down Expand Up @@ -311,10 +312,13 @@ Reference finder for rename planning. Input: `{ oldName: string, newName: string
## 16. get_processes
Entry point execution flows. Input: `{ entryPoint?: string, limit?: number }`. Returns: processes with steps and depth.

## 17. get_clusters
## 17. analyze_highways
Repeated route convergence. Input: `{ operation?: string, shape?: string, minRoutes?: number, propose?: boolean, trace?: string }`. Returns: bypass/cowpath opportunities with route chains, evidence, blast radius, proposed canonical node, and context pack.

## 18. get_clusters
Community-detected file clusters. Input: `{ minFiles?: number }`. Returns: clusters with cohesion.

## 18. check
## 19. check
Rules-engine gate. Input: `{}`. Returns: pass/warn/fail verdict, findings, suppression ledger, config path, and summary counts. Findings always include ruleId, severity, file, line, column, message, and fingerprint; cleanup findings may also include kind, confidence, and evidence. Suppressions include directive, active/stale status, file, line, targetLine, ruleIds, and suppressed count.

## Tool Selection Guide
Expand All @@ -338,14 +342,15 @@ Rules-engine gate. Input: `{}`. Returns: pass/warn/fail verdict, findings, suppr
| What changed? | detect_changes |
| Find all references to X | rename_symbol |
| How does data flow? | get_processes |
| Which routes bypass canonical dataflow? | analyze_highways |
| What files naturally belong together? | get_clusters |
| What rule violations exist? | check |

---

# CLI Reference

19 commands — 17 analysis commands (full parity with MCP tools), `check` for CI rules, and `init` for agent adoption.
20 commands — 18 analysis commands (full parity with MCP tools), `check` for CI rules, and `init` for agent adoption.

## Commands

Expand Down Expand Up @@ -445,6 +450,12 @@ codebase-intelligence processes <path> [--entry <name>] [--limit <n>] [--json] [
```
Entry point execution flows through the call graph.

### highways
```bash
codebase-intelligence highways <path> [--operation <verb>] [--shape <name>] [--min-routes <n>] [--propose] [--trace <id>] [--json] [--force]
```
Find repeated routes that should converge on one canonical operation path. Returns bypass/cowpath findings with route chains, canonical node, evidence, blast radius, recommendation, and context pack.

### clusters
```bash
codebase-intelligence clusters <path> [--min-files <n>] [--json] [--force]
Expand Down
5 changes: 3 additions & 2 deletions llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
- [Architecture](docs/architecture.md): Pipeline, module map, data flow, design decisions
- [Data Model](docs/data-model.md): All TypeScript interfaces with field descriptions
- [Metrics](docs/metrics.md): Per-file and module metrics, force analysis, complexity scoring
- [MCP Tools](docs/mcp-tools.md): 18 MCP tools — inputs, outputs, use cases, selection guide
- [MCP Tools](docs/mcp-tools.md): 19 MCP tools — inputs, outputs, use cases, selection guide
- [CLI Reference](docs/cli-reference.md): CLI commands, flags, output formats, examples

## Quick Start
Expand All @@ -17,7 +17,7 @@ MCP mode (AI agents):
codebase-intelligence ./path/to/project
```

CLI mode (humans/CI) — 19 commands (17 analysis with MCP parity + `check` + `init`):
CLI mode (humans/CI) — 20 commands (18 analysis with MCP parity + `check` + `init`):
```bash
codebase-intelligence overview ./src
codebase-intelligence hotspots ./src --metric coupling
Expand All @@ -35,6 +35,7 @@ codebase-intelligence symbol ./src parseCodebase
codebase-intelligence impact ./src getUserById
codebase-intelligence rename ./src oldName newName
codebase-intelligence processes ./src --entry main
codebase-intelligence highways ./src --operation create --min-routes 3 --json
codebase-intelligence clusters ./src --min-files 3
codebase-intelligence check ./src # rules gate for CI; JSON includes findings plus active/stale suppressions
codebase-intelligence init . # make AI agents use CI
Expand Down
15 changes: 9 additions & 6 deletions roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -387,17 +387,20 @@ entry ─► transform ─► validate ─► sink
| `bypass` | Route that reaches a sink while skipping an existing canonical node |
| `highway` | Approved canonical route for one `(operation, shape, sink)` inside a bounded scope |

**To do:**
**H1 slice:**

- H1 shipped: classify operation verbs, enumerate entry-to-sink routes, detect cowpaths and bypasses.
- H1 shipped: propose reroutes to existing canonical nodes with route chains, blast radius, evidence, and context packs.
- MCP shipped: add `analyze_highways`.
- CLI shipped: add `highways <path>` with `--operation`, `--shape`, `--min-routes`, `--propose`, `--trace`, `--json`.
- H1 shipped: every opportunity emits a token-budgeted context pack: summary, affected routes, evidence, blast radius, proposed canonical node, next safe command.

**Remaining:**

- H1: classify operation verbs, enumerate entry-to-sink routes, detect cowpaths and bypasses.
- H1: propose reroutes to existing canonical nodes.
- H2: add type-shape grouping once Type/Shape layer lands.
- H2: detect shape drift and near-duplicate intermediate steps.
- H2: synthesize new highway proposals: name, location, signature, skeleton, cycle-safe reroute plan.
- H3: add reuse hotspot metrics and cross-link opportunities into `forces` / `hotspots`.
- MCP: add `analyze_highways`.
- CLI: add `highways <path>` with `--operation`, `--shape`, `--min-routes`, `--propose`, `--trace`, `--json`.
- Every opportunity should emit a token-budgeted context pack: summary, affected routes, evidence, blast radius, proposed canonical node, next safe command.

### Scope Graph + Codebase Map

Expand Down
Loading
Loading