Feature request
Improve semantic_query retrieval quality with two independent, incremental steps:
-
Hybrid retrieval: fuse the existing FTS5/BM25 path with the vector path via Reciprocal Rank Fusion (k≈60). Today BM25 (query) and vectors (semantic_query) are separate modes; RRF-merging them is ~100 lines, deterministic, and in three surveyed implementations it materially rescued weak vector ranking. Care needed on the join key (one surveyed implementation fused on mismatched keys, silently disabling the fusion).
-
Include function/method bodies in the embedded text (or an option to). Today node vectors are built from name/signature/docstring metadata only, so semantic_query can't match on what code does, only what it's called. The one surveyed tool that embeds actual source bodies (with a real ANN index) measured retrieval F1 0.912 vs 0.381–0.536 for grep on the same oracle — the only credible head-to-head in the category. Bodies could be token-composed through the existing int8 table + RI fallback without shipping a runtime transformer; even truncated at 512 tokens the signal gain is large.
Independent of both: the brute-force cbm_cosine_i8 full scan over node_vectors is fine at 50k nodes but linear at kernel scale — worth a note in docs, or an eventual ANN structure.
Context: same 10-tool survey as the sibling issues; the maintainer-facing summary is that metadata-only embedding is the single most common cause of "semantic search returned nothing relevant" reports across this tool category.
Feature request
Improve
semantic_queryretrieval quality with two independent, incremental steps:Hybrid retrieval: fuse the existing FTS5/BM25 path with the vector path via Reciprocal Rank Fusion (k≈60). Today BM25 (
query) and vectors (semantic_query) are separate modes; RRF-merging them is ~100 lines, deterministic, and in three surveyed implementations it materially rescued weak vector ranking. Care needed on the join key (one surveyed implementation fused on mismatched keys, silently disabling the fusion).Include function/method bodies in the embedded text (or an option to). Today node vectors are built from name/signature/docstring metadata only, so
semantic_querycan't match on what code does, only what it's called. The one surveyed tool that embeds actual source bodies (with a real ANN index) measured retrieval F1 0.912 vs 0.381–0.536 for grep on the same oracle — the only credible head-to-head in the category. Bodies could be token-composed through the existing int8 table + RI fallback without shipping a runtime transformer; even truncated at 512 tokens the signal gain is large.Independent of both: the brute-force
cbm_cosine_i8full scan overnode_vectorsis fine at 50k nodes but linear at kernel scale — worth a note in docs, or an eventual ANN structure.Context: same 10-tool survey as the sibling issues; the maintainer-facing summary is that metadata-only embedding is the single most common cause of "semantic search returned nothing relevant" reports across this tool category.