Skip to content

fix(rag): tokenize CJK text so BM25 retrieval can score non-Latin queries - #146

Closed
kuroudo-ai wants to merge 1 commit into
LLMQuant:masterfrom
kuroudo-ai:fix/bm25-cjk-tokenization
Closed

fix(rag): tokenize CJK text so BM25 retrieval can score non-Latin queries#146
kuroudo-ai wants to merge 1 commit into
LLMQuant:masterfrom
kuroudo-ai:fix/bm25-cjk-tokenization

Commits

  1. Commits on Aug 13, 2026