Skip to content

feat(explore): add compact map modes and make map the default - #615

Merged
bompus merged 7 commits into
fork/consolidatedfrom
feat/explore-map
Nov 1, 2026
Merged

bompus merged 7 commits into
fork/consolidatedfrom
feat/explore-map

Conversation

@bompus

@bompus bompus commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

What

codegraph_explore returns a compact map by default: matching symbols with file:line, signatures, the call path, dynamic boundaries and blast radius, without source bodies. mode: "snippets" keeps each file's first 12 lines; mode: "source" is byte-for-byte the previous output.

The served surface follows the mode: map/snippets list codegraph_node as the body follow-up, and node is marked anthropic/alwaysLoad so it is callable without a ToolSearch step. Source mode keeps the old two-tool surface.

Precedence: per-call mode → CODEGRAPH_EXPLORE_MODE → codegraph.json explore.mode → map. The CLI gains --mode.

Evidence (2026-11-01, 3-turn sessions, same questions)

corpus map residual source residual no codegraph
gin 8,369 tok 13,327 5,268
n8n 20,430 28,806 21,882

Map is 37% / 29% below source. On n8n, source is +11% over its no-codegraph control while map is -6.6%. Single run per arm; the map arm predates the node always-load fix, so it is a lower bound. A first pass was discarded because a persistent MCP daemon kept its startup mode; the clean run used CODEGRAPH_NO_DAEMON=1.

Also

  • Map responses no longer promise source they omit; the Source Code header says it is omitted; a fourth relationship kind can no longer land under the third.
  • codegraph_node's guidance names includeCode: true (its default is false).
  • README, site CLI/MCP pages, installer instructions and CHANGELOG updated.
  • Existing rendering tests are pinned to source in vitest.config.mts; __tests__/explore-compact.test.ts covers modes, the surface and the default.

Tests

  • Full suite: 961 files, 11,380 passed, 46 skipped.
  • check:agent-docs clean.
  • README rows checked: "What the fork adds" compact-explore row, CLI codegraph explore, MCP Tools table; site reference/cli.md and reference/mcp-server.md.

The map is the compact answer: symbols, signature, file:line, flow and
blast radius without source bodies. Snippets keep each file's first lines;
source is the previous output. The served surface follows the resolved
mode so map mode lists codegraph_node as its body-on-demand path.
A map no longer promises source it omits, the Source Code header says it is
omitted, and a fourth relationship kind can no longer land under the third.
The CLI mirrors the MCP mode argument, and codegraph_node is marked
always-load so map mode's follow-up does not need a ToolSearch step.
The 2026-11-01 three-turn A/B cut retrieval residual 29-37% against
source mode on gin and n8n. Existing rendering tests are pinned to
source; the map default and its surface have their own tests.
@bompus bompus added the author:deepseek Pull request written by a deepseek model label Nov 1, 2026

@bompus-ai-review bompus-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bompus AI Review (GPT-6.1 Terra, max reasoning): 8 finding(s).

Comment thread src/mcp/tools.ts Outdated
if (mode === 'snippets') out.push(line);
continue;
}
if (line.startsWith('**`')) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high: Build the map from symbol locations and signatures

The map keeps each file header with out.push(line) but drops every source-fence line. The existing whole-file header contains only ${n.name}(${n.kind}), not the symbol's line or signature.

Failure: A query for a leaf function with no callers or multi-symbol flow returns a header such as src/a.ts — leaf(function) but no src/a.ts:42 or function signature. Those are part of the advertised default map, so an ordinary location or signature question cannot be answered from the result.

Suggested fix: Render each selected symbol's indexed location and signature explicitly before discarding its source.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: compact maps now inject a - name (kind) :line — signature row for each selected symbol under its file header before the source is dropped. Covered by the symbol-row and injection tests.

Comment thread src/mcp/tools.ts Outdated
}
continue;
}
if (line === '---') out.push(line);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Preserve graph sections that follow documentation source

After **Source Code**, inSource never becomes false, and the outside-fence branch discards every line other than file headers, pointers, and ---. For mixed documentation/code results, the renderer appends graphLines after the source sections.

Failure: A query naming both README.md and a code symbol can render both files, but map and snippets modes silently remove the code symbol's blast radius and relationships.

Suggested fix: Recognize the end of the source sections and preserve or separately render the trailing graph sections.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: the compact pass tracks Source Code, blast radius, discussed, relationships, pointers and the preamble explicitly, so graph sections that follow documentation source are preserved. Covered by the doc-tier tail test.

Comment thread src/mcp/tools.ts
}
continue;
}
if (line.startsWith('- ')) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Report relationships omitted by the compact limits

The compact pass emits only entries for which relEntries < 4 and silently skips later relationship kinds. It also drops the original renderer's ... and N more entry when that entry follows the first four.

Failure: In a repository where a result has five or more call edges, the map shows four under calls with no indication that other calls were omitted. A caller can reasonably mistake that list for the complete relationship set.

Suggested fix: Count and report omitted entries and relationship kinds, including any omissions already reported by the source renderer.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: dropped relationship entries and kinds are counted and reported, including the renderer's own ... and N more, so the compact list no longer reads as complete. Covered by the omission test.

Comment thread src/mcp/tools.ts
export function resolveExploreMode(
arg: unknown,
projectRoot: string,
env: NodeJS.ProcessEnv = process.env,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Apply the launching session's mode override through the daemon

resolveExploreMode reads process.env, while an MCP proxy forwards tool calls to a shared daemon that inherited its environment when it was first spawned.

Failure: If a daemon was started in map mode, launching a later session with CODEGRAPH_EXPLORE_MODE=source still produces map responses until that daemon exits. The proxy meanwhile uses the later session's environment for its tool list, so the advertised surface can also disagree with the response.

Suggested fix: Pass the effective session override to the daemon and query worker, and resolve it before the project configuration.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: the proxy sends CODEGRAPH_EXPLORE_MODE in its client hello; the daemon carries it per session, and resolveExploreMode applies it before the daemon's inherited env and the project config. Covered by the hello and precedence tests.

Comment thread src/mcp/tools.ts Outdated
+ '\n\n... (compact response truncated; narrow the query or name the symbols you need)';
}
const hint = mode === 'map'
? '> Source omitted in compact map mode. For one symbol\'s body, call codegraph_node with its name and `includeCode: true`; for the full source, call codegraph_explore with mode: "source".'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Only recommend a body follow-up that the client can call

Every compact response says to call codegraph_node, but defaultSurfaceFor('source') omits it and an explicit CODEGRAPH_MCP_TOOLS=explore allowlist disables it.

Failure: With CODEGRAPH_EXPLORE_MODE=source, a per-call mode: "map" returns a map directing the client to a tool absent from its registered surface. With the explore-only allowlist, the same suggested call is rejected as disabled.

Suggested fix: Make node available when compact output can be requested, or tailor the response to direct clients to an available source-mode follow-up.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: the response derives nodeServed from the effective mode plus the allowlist and only names codegraph_node when it is on that surface; otherwise it points at mode: "source". Covered by the surface tests.

Comment thread src/mcp/tools.ts
let body = out.join('\n').replace(/\n{3,}/g, '\n\n');
const maxChars = mode === 'map' ? EXPLORE_MAP_MAX_CHARS : EXPLORE_SNIPPETS_MAX_CHARS;
if (body.length > maxChars) {
const cut = body.slice(0, maxChars - 160);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Keep the query echo from consuming the entire map budget

The compact cap slices the rendered answer from its beginning, and that answer includes the complete query in the **Exploration: ${query}** heading. Queries up to 10,000 characters are accepted.

Failure: For a valid single-line query longer than about 3,840 characters that names an indexed file, the truncated map contains only the echoed query and truncation hint; the found-symbol summary and file headers never reach the caller.

Suggested fix: Bound or omit the echoed query before applying the compact budget, reserving space for the actual results.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: the **Exploration: echo is bounded to 160 characters before the compact cap, so it cannot consume the map. Covered by the long-query test.

Comment thread src/mcp/tools.ts Outdated
out.push(line);
continue;
}
if (inBlast) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: End the blast-radius limit before the session-history section

inBlast is reset only by **Relationships** or **Source Code**. The renderer places **Discussed in earlier sessions** immediately after the blast radius, so its bullet points also consume blastEntries.

Failure: When five symbols have blast-radius entries and several were discussed previously, only the first session-history row survives; the other recorded discussions disappear without an omission notice.

Suggested fix: End blast-radius processing at the next section heading and apply its limit only to blast-radius entries.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: the blast-radius limit now ends at the next section heading, so **Discussed in earlier sessions** keeps its rows. Covered by the blast/session test.

Comment thread src/mcp/tools.ts Outdated
}
continue;
}
if (mode === 'snippets' && snippet < EXPLORE_SNIPPET_LINES) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Return the promised file beginnings in snippets mode

Snippets mode keeps the first 12 lines of each already-rendered fence. Those fences can start at a selected documentation section or a clustered symbol range rather than at line 1 of the file.

Failure: For a matched README section beginning at line 87, snippets mode returns lines beginning at 87 while claiming to show the first 12 source lines of the file. Large code files can produce the same mismatch for a selected function.

Suggested fix: Construct snippets from the first 12 on-disk lines of each selected file, or change the documented contract and response hint to describe the selected excerpts accurately.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 36ee5a2: the snippets contract and hint now say "the first 12 lines of each returned excerpt", matching the selected-section behavior; README, site and changelog wording updated.

Map rows carry the selected symbol's location and signature; sections are
tracked explicitly so doc-tier graph tails survive; relationship and blast
omissions are reported; the launching session's mode travels through the
daemon; node is only named when served; the query echo and snippets contract
are bounded.

@bompus-ai-review bompus-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bompus AI Review (GPT-6.1 Terra, max reasoning): 9 finding(s).

Comment thread src/mcp/tools.ts Outdated
if (!group) continue;
// Doc headings are navigation, not code to locate; a smaller list keeps
// the doc-tier graph tail inside the map budget.
const rows = compactSymbolRows(group.nodes, docTierFiles.has(fp) ? 3 : 6);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high: Prioritize explicitly requested symbols in compact rows

The map builds rows with compactSymbolRows(group.nodes, ... ? 3 : 6), and that function stops after the first six eligible nodes. Pinned-file nodes are inserted in source order before line anchors are resolved.

Failure: For a small file containing seven independent functions, a query for file.ts:line inside the seventh can return only the first six functions. With no callers or flow, the requested function's name, location and signature disappear when its source is removed. The earlier symbol-row fix still does not cover every selected symbol.

Suggested fix: Order exact line-anchor and named targets ahead of other nodes before applying the row limit, and report any omitted rows.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: compactSymbolRows now orders query roots, line anchors, flow-spine and named nodes ahead of source order, then reports how many eligible rows the cap omitted. Covered by the priority test.

Comment thread src/mcp/tools.ts Outdated
if (!group) continue;
// Doc headings are navigation, not code to locate; a smaller list keeps
// the doc-tier graph tail inside the map budget.
const rows = compactSymbolRows(group.nodes, docTierFiles.has(fp) ? 3 : 6);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Preserve quoted-source match locations in maps

The quoted-prose scan adds matches to lineAnchors, but compact tables are built only from group.nodes, and map mode removes the source fence containing the match.

Failure: A quoted phrase on line 80 of a small indexed template with no graph nodes produces a file header but no match line or symbol row. The caller cannot locate the occurrence from the default answer, even though the scan found it.

Suggested fix: Include bounded file:line entries for quoted-source matches in compact output, including files without symbols.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: each quoted/path line anchor adds a bounded - line :N row under its file header, even when the file has no symbol nodes.

Comment thread src/mcp/tools.ts Outdated
// is only the fallback when the launching session had none.
const mode = resolveExploreMode(args.mode, projectRoot, process.env, args.__exploreMode ?? sessionMode);
const allow = this.toolAllowlist();
const nodeServed = allow ? allow.has('node') : defaultSurfaceFor(mode).has('node');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Base body-follow-up guidance on the advertised tool surface

nodeServed uses defaultSurfaceFor(mode), where mode is the current call's mode; the proxy's tools/list instead uses the session environment through getStaticTools().

Failure: With CODEGRAPH_EXPLORE_MODE=source, the proxy lists only explore and sessions. A per-call mode: "map" nevertheless sets nodeServed to true and directs the client to the unlisted codegraph_node. The prior fix covers the allowlist but still does not use the surface actually advertised to this client.

Suggested fix: Carry the session's advertised surface into the handler and use it for follow-up guidance; keep that surface consistent with project configuration as well.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: the proxy sends its advertised surface mode in the client hello; nodeServed now uses that surface instead of the per-call mode, so a map requested from a source-only session no longer recommends node.

Comment thread src/mcp/tools.ts Outdated
): ExploreMode {
return exploreModeOption(arg)
?? exploreModeOption(sessionMode)
?? exploreModeWithoutProject(env)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Do not inherit an old daemon's mode when a later session unsets it

sendClientHello omits exploreMode when the launching environment has no value, and resolveExploreMode then falls through to exploreModeWithoutProject(env), which reads the daemon's inherited environment.

Failure: Start a shared daemon with CODEGRAPH_EXPLORE_MODE=source, then launch another session with the variable unset and a project configured for map. The later session still receives source until the daemon exits, despite the documented session environment and project precedence. The earlier fix handles an explicitly supplied override, but not its absence.

Suggested fix: Send an explicit per-session indication that the override is unset and skip the daemon's inherited override for that session.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: the hello always carries exploreMode (null when the session has none). A null session skips the daemon's inherited env and falls through to the project config/default; covered by the null-session precedence test.

Comment thread src/mcp/tools.ts
}
continue;
}
if (line.trim()) out.push(line);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Remove legacy source-completeness claims from compact output

The pointers section copies every nonblank line, including the source renderer's Complete source for ... is included above epilogue. Separately, the dynamic-boundary source promise is rewritten only when mode === 'map'.

Failure: A map with a trailing pointer list can say that complete source is included and should not be reread after all source fences were removed. In snippets mode, a dispatch site after the first 12 lines can likewise be absent while the response says its source is included below.

Suggested fix: End pointer processing at the epilogue and remove or rewrite source-completeness promises for both compact modes.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: compact output drops the "Complete source ... included above" claims, and the snippets dispatch promise now says "excerpted below" so a site beyond the excerpt is not implied to be present.

Comment thread src/mcp/tools.ts Outdated
for (const line of text.split('\n')) {
const relKindLine = /^\*\*[^*\n]+:\*\*$/.test(line);
if (!inFence) {
if (line.startsWith('**Exploration: ') && line.endsWith('**')) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Bound multiline query echoes before applying the compact cap

The 160-character limit applies only when one line both starts with **Exploration: and ends with **; accepted queries may contain newlines.

Failure: A valid query consisting of src/a.ts, a newline and 4,000 more characters bypasses that limit. The compact cap then cuts the echoed query before the found-symbol summary and file header, returning no result for the pinned file. The earlier long-query fix only covers a single-line heading.

Suggested fix: Bound or omit the complete query before interpolating it into the rendered heading, including multiline queries.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: the renderer normalizes and truncates the query before interpolating the heading (multiline included), and the compactor accumulates a multiline echo before bounding it.

Comment thread src/mcp/tools.ts Outdated
}

if (inFence) {
if (line.startsWith('```')) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Distinguish nested Markdown fences from the source wrapper

While inside a source fence, any line starting with triple backticks sets inFence = false, including a Markdown excerpt's nested ```yaml opener.

Failure: With CODEGRAPH_EXPLORE_LINENUMS=0, a selected Markdown section containing a fenced YAML list causes list items to be emitted as compact symbol rows. A long list can consume the map cap and remove graph sections that follow the documentation source.

Suggested fix: Compact from structured source sections, or track wrapper and nested Markdown fences separately when line numbering is disabled.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: source fences are chosen longer than any backtick run in the body, and the compactor tracks the wrapper length and only closes on a bare fence at least that long. Covered by the nested-fence test.

Comment thread src/mcp/tools.ts
body = (at > 0 ? cut.slice(0, at) : cut)
+ '\n\n... (compact response truncated; narrow the query or name the symbols you need)';
}
const followUp = nodeServed

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Use a body follow-up that works for container symbols

The compact hint says to call codegraph_node with a symbol name and includeCode: true for its body, but renderNodeSection returns buildContainerOutline instead of source for classes, modules and other containers with indexed children.

Failure: After a map identifies a class, following the advertised call returns only member names and signatures, not the class body needed to inspect or edit it. Markdown sections with nested headings have the same problem.

Suggested fix: Offer a full-body node option, or direct container and Markdown-section follow-ups to a source-producing call.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: the hint and tool description now say a container returns its member outline and point at mode: "source" for its full body.

Comment thread src/mcp/tools.ts
if (line === '**Source Code**') {
flushOmissions();
section = 'source';
out.push(mode === 'map'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium: Make the retrieval replay recognize maps without source

Map mode emits **Source Code** — omitted in map mode, while probe-explore.mjs detects source with text.indexOf('**Source Code**') and counts file headers as rendered source files.

Failure: Comparing a source answer with a map containing the same file headers reports lostSource: false and no lost files; with the current corpus's empty mustMatch lists, replay reports the answer as changed without loss despite every body being absent. This defeats the required retrieval regression gate's source-loss check.

Suggested fix: Make the probe and replay mode-aware, distinguishing omitted source from rendered source and checking source-mode compatibility alongside compact-map coverage.

@bompus bompus Nov 1, 2026 •

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fbd13da: probe-explore.mjs is mode-aware — maps report sourceAt: -1, files: [], mapFiles: [...] and compact: true — and defaults to --mode source so the replay source-loss gate compares bodies. Covered by the probe-row test.

Prioritize requested symbols and report omitted rows; carry quoted/path
anchors into maps; use the advertised surface for the body follow-up; send
an explicit unset session mode; drop source-completeness claims; bound
multiline echoes; size fences for nested Markdown; point containers at
source mode; and make the replay probe mode-aware.

@bompus-ai-review bompus-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bompus AI Review (GPT-6.1 Terra, max reasoning): 0 finding(s).

Review 3: only high findings and above are posted from here on.

@bompus
bompus merged commit 8ca0a91 into fork/consolidated Nov 1, 2026
3 checks passed
@bompus
bompus deleted the feat/explore-map branch November 1, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author:deepseek Pull request written by a deepseek model

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant