Skip to content

feat(search): add bounded archive discovery with eligible replies - #803

Merged
leemour merged 14 commits into
mainfrom
feat/combined-search
Oct 10, 2026
Merged

leemour merged 14 commits into
mainfrom
feat/combined-search

Conversation

@leemour

@leemour leemour commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Natural questions require words that the existing strict matcher cannot find together, and short replies often lack the query's words entirely. Add explicit archive discovery through CLI search messages --discover, MCP discover=true and SDK SearchQuery.discover. Partial lexical retrieval uses existing SQLite FTS/BM25, ranks bounded evidence with stemmed term coverage and reciprocal retrieval rank, and recovers eligible direct replies. A reply inherits its parent's lexical rank and comes first on an equal score. No neural assets, full-archive snapshot or automatic conversation rebuild is used.

Hard account/chat/sender/date/only constraints apply to both parent and reply. Explicit Boolean syntax, phrases, wildcards, AST, exact and newest requests retain strict behavior. Matched/missing terms, reply-parent provenance, candidate depth and truncation make partial evidence visible. Discovery searches the archive, refuses legacy/regex/server-only modes, and preserves requested surrounding/thread context. Saved runs keep the additive option. Plain keyword discovery also preserves used prefix/correction provenance.

Migration 29 adds a derived live-reply index, reserved by merged PR #817; no message data changes. Rewrite only selection into indexed message-key lookups, avoiding a correlated JSON test over every archive row. Retrieve at most 300 lexical candidates and 100 proposed replies, then keep at most 300 scoring candidates under existing body/time budgets. A full-shortlist regression test ensures replies can enter the final pool.

The original one-default proposal remains unqualified by its approved quality gates. This is an additive opt-in; strict CLI/MCP defaults and no-language SDK compatibility remain unchanged. The source handoff and query guide describe this boundary. Synthetic research, original artifacts, the ranking matrix and fresh/scaling reports live in WireCatLabs/cli-testing PR #5. The freshly authored fixture is not an independent human/real-world evaluation; rank one and paraphrases remain limitations. Scores are not answer confidence.

Validation: lint, typecheck, 2790 passing tests (2 existing skips), coverage floors, documentation checks and Bun smoke pass. Isolated SDK tests cover scope/account/deletion, full candidate pools, syntax/AST/flags, contexts, typo provenance, cancellation and bounds. Telegram and MAX offline adoption tests are being verified against a packed build before the shared release and exact consumer pin bumps. No real account/store/messenger operation was performed.

@leemour leemour changed the title test(search): measure message ranking and propose combined search feat(search): evaluate combined lexical ranking before changing defaults Oct 9, 2026
@leemour
leemour force-pushed the feat/combined-search branch from 9259bfb to 99344d8 Compare October 9, 2026 12:31
@leemour leemour changed the title feat(search): evaluate combined lexical ranking before changing defaults test(search): measure combined candidates and local joint reranking Oct 9, 2026
@leemour leemour changed the title test(search): measure combined candidates and local joint reranking feat(search): prototype combined message candidate retrieval Oct 9, 2026
@leemour
leemour force-pushed the feat/combined-search branch from ac68a77 to 2452271 Compare October 10, 2026 13:44
@leemour leemour changed the title feat(search): prototype combined message candidate retrieval feat(search): add bounded archive discovery with eligible replies Oct 10, 2026
@leemour
leemour marked this pull request as ready for review October 10, 2026 13:45
Comment thread src/search/question-plan.ts Fixed
Comment thread src/search/question-plan.ts Fixed
@leemour
leemour merged commit 59c2fc7 into main Oct 10, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants