From 44573f06e812620aff62128e4ba2c7dad445eec9 Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 13:30:07 +0200 Subject: [PATCH 01/14] test(search): measure message ranking and propose combined search --- bench/message-search-quality/README.md | 77 + bench/message-search-quality/baseline.json | 11549 +++++++++++ bench/message-search-quality/corpus.json | 19452 +++++++++++++++++++ bench/message-search-quality/generate.py | 60 + bench/message-search-quality/run.ts | 130 + bench/message-search-quality/tsconfig.json | 5 + docs/dev/combined-search.md | 183 + 7 files changed, 31456 insertions(+) create mode 100644 bench/message-search-quality/README.md create mode 100644 bench/message-search-quality/baseline.json create mode 100644 bench/message-search-quality/corpus.json create mode 100644 bench/message-search-quality/generate.py create mode 100644 bench/message-search-quality/run.ts create mode 100644 bench/message-search-quality/tsconfig.json create mode 100644 docs/dev/combined-search.md diff --git a/bench/message-search-quality/README.md b/bench/message-search-quality/README.md new file mode 100644 index 000000000..b9ed0df41 --- /dev/null +++ b/bench/message-search-quality/README.md @@ -0,0 +1,77 @@ +# Message search ranking quality + +This offline benchmark compares the production strict Lucene search (including its default +stemming) and legacy fallback chain through `searchStore`. It contains 2,616 invented messages +in nine chats, and 96 labelled queries: 40 answerable and eight no-answer per split. Each topic +has two useful messages and 300 newer distractors. The busy operations chat holds 1,600 topic +distractors plus 200 unrelated messages; eight topic chats each hold 102 messages. + +The committed [corpus.json](corpus.json) is the evaluation input. [generate.py](generate.py) +reproduces it deterministically with no packages or network. No real account, messenger adapter, +model cache or network request is used. The runner opens an explicit new temporary SQLite file, +points `MESSAGING_STORE` and the model cache into that temporary directory, builds both indexes, +and closes the store in `finally`. It keeps the temporary files for inspection; removal is a +separate owner-approved cleanup task. + +## Reproduce + +From the repository root, with dependencies installed: + +```sh +pnpm build +pnpm exec tsc -p bench/message-search-quality/tsconfig.json +node bench/message-search-quality/run.ts > bench/message-search-quality/baseline.json +``` + +The script fixes UTC for both dialects. It exits unsuccessfully for invalid labels, duplicate +message/query ids, incomplete indexes, unknown result ids, duplicate hits or a missing measured +query/mode. Every query produces numeric metrics in both modes; answerable metrics are null +only for no-answer queries. The aggregate reports every split, mode and category. Corpus, runner +and relevant production-file hashes identify what was measured; the temporary path changes per run. +The benchmark runs outside `pnpm test`; its own TypeScript check is required because the repository +lint and TypeScript configuration exclude this folder. + +## Labels and limits + +Queries cover whole words, typos, word beginnings, Cyrillic and Latin, phrases, sender/chat/date +filters, AND/OR, paraphrases, and absent/near-miss no-answer requests. Each query states its intent. +Grade 2 is the message answering the decision/policy question; grade 1 is its useful owner or +confirmation message. Templates, unanswered questions and rejected proposals are grade 0. +The sender-filter queries label only the grade-2 message because the confirmation has another sender. +All other answerable queries label both useful messages. No-answer requests have no labels. + +Lucene `date:2026-10-02` is paired with legacy `after:2026-10-02 before:2026-10-03`. +Explicit Lucene AND is paired with legacy implicit conjunction. These are handwritten equivalents; +the runner does not reinterpret an unsupported dialect silently. OR is limited to the same +precedence-compatible shape in both parsers. The `exact` flag is not enabled: the Lucene baseline +measures the existing default strict service, with stems ready. + +Dev topics are Aurora, Cedar, Север and Lumen; held-out topics are Harbor, Maple, Вектор and Prism. +The topic families and labelled messages do not overlap, but both are indexed in the same archive. +Templates and query categories are shared. This is a small synthetic holdout, not a blinded external +study or evidence of general multilingual quality. Held-out results are reported for baseline +transparency; future parameter choices must use dev only. A lexical reranker cannot reliably +recognize an agreed decision merely from shared topic words. Paraphrases remain explicit misses +in a lexical-only release; do not redefine their labels to inflate its score. + +Recall@10 is the fraction of labelled messages in the first ten hits, macro-averaged over +answerable queries. MRR uses the first grade-1-or-2 hit within the returned ten (zero for a miss). +nDCG@10 uses gain `2^grade - 1` and a discount of `log2(rank + 1)`, normalized against the sorted +labels. No-answer false hits count all returned messages on no-answer queries, up to ten each; +query hit rate is the fraction of no-answer queries returning any hit. These are retrieval false +hits, not claims about an agent's eventual answer. Raw ids, labels, match types, scores and +corrections are retained in [baseline.json](baseline.json), without real messages. + +## Baseline + +| Split | Mode | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | No-answer query hit rate | +|---|---|---|---|---|---|---| +| dev | Lucene | 0.313 | 0.325 | 0.287 | 0 / 80 | 0.000 | +| dev | legacy | 0.388 | 0.425 | 0.354 | 80 / 80 | 1.000 | +| held-out | Lucene | 0.263 | 0.325 | 0.219 | 0 / 80 | 0.000 | +| held-out | legacy | 0.338 | 0.450 | 0.299 | 80 / 80 | 1.000 | + +The strict matcher misses typos and word beginnings. Legacy recovers some, but its early successful +list prevents other matchers from contributing; broad any-word fallback fills no-answer pages. +Both can bury useful messages beneath exact keyword matches. Sender/date filters are easy controls +and achieve full recall; category reports prevent them from concealing poor unfiltered rankings. diff --git a/bench/message-search-quality/baseline.json b/bench/message-search-quality/baseline.json new file mode 100644 index 000000000..2bd5d44ca --- /dev/null +++ b/bench/message-search-quality/baseline.json @@ -0,0 +1,11549 @@ +{ + "schemaVersion": 1, + "k": 10, + "node": "v24.19.0", + "timezone": "UTC", + "storePath": "/tmp/message-search-quality-XIyNa5/quality.db", + "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", + "runnerSha256": "ff9cbc4de7a2f6312ba6a2f3565bcda9b6b4105f3c50fc65cbbd7f8ffc1c186c", + "productionSha256": { + "services/messages.js": "62f927cddd912275bb46a7c990d64881c2b4a22eac12d54f484fd5ca4223a3e4", + "services/messages-search.js": "35eb4ed1dcc002a80985327ffa9f6bd25a5ee6b1a31f3024203a38b6605c7d2a", + "search/search.js": "e1c7975b1573936515244fdc60454a699552afde89363b517d35974e115384b6", + "search/query.js": "89a7030d749f06ca61b9867bf29db4244617943e14c383aa5a09fd40ed819772", + "search/correct.js": "54e5089f217840dd9e27e6e89c29486c2d8704850af16dafd1b8410998066270", + "search/stem.js": "84475baa966a17595c480015668dc77da60409a00c391aa9ca6a718433386007", + "search/lucene/parser.js": "28f9a1edbeb172e4556fcc23528c83e2acf008ff70dae48ca99dbcfe62b1531d", + "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964", + "store/sqlite/words.js": "be7c86aa22f8a308ce2fba664801ffd65da4aad444f1a8e62533548980a9e5b3" + }, + "messages": 2616, + "queries": 96, + "metrics": { + "dev": { + "lucene": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3125, + "mrr": 0.325, + "ndcgAt10": 0.2865560469084554, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.25, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.9491768952476267, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + } + } + }, + "legacy": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3875, + "mrr": 0.425, + "ndcgAt10": 0.3539325811955994, + "noAnswerFalseHits": 80, + "noAnswerQueryHitRate": 1, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.25, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.9491768952476267, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.5, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + } + } + } + }, + "test": { + "lucene": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.2625, + "mrr": 0.325, + "ndcgAt10": 0.21917951262131133, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.8475306857428799, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + } + } + }, + "legacy": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3375, + "mrr": 0.45, + "ndcgAt10": 0.2992618230965487, + "noAnswerFalseHits": 80, + "noAnswerQueryHitRate": 1, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.8475306857428799, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.5, + "mrr": 0.75, + "ndcgAt10": 0.66311732856428, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + } + } + } + } + }, + "rankings": [ + { + "id": "aurora-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "aurora rolluot rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "aurora rolluot rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10168", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10163", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10158", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10153", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10148", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10143", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10138", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10133", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10128", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + } + ], + "corrections": [ + { + "from": "rolluot", + "to": [ + "rollout" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "aurora roll rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "aurora roll rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10168", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10163", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10158", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10153", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10148", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10143", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10138", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10133", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10128", + "chatId": "999", + "match": "beginnings", + "score": 7.447603932068213 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"aurora rollout rollback\"", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"aurora rollout rollback\"", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "aurora rollout from:700", + "relevant": { + "10001": 2 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": null, + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "aurora rollout from:700", + "relevant": { + "10001": 2 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": "words", + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "aurora rollout chat:101", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "aurora rollout chat:101", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10293", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10288", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10283", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10278", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10273", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10268", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10263", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10258", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10253", + "chatId": "101", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "aurora rollout date:2026-10-02", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": null, + "score": 4.340227569072966 + }, + { + "id": "10002", + "chatId": "101", + "match": null, + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "aurora rollout after:2026-10-02 before:2026-10-03", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": "words", + "score": 4.340227569072966 + }, + { + "id": "10002", + "chatId": "101", + "match": "words", + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "aurora AND rollout AND rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "aurora rollout OR rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "aurora rollout OR rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "aurora deployment reversal", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "aurora deployment reversal", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "anyWord", + "score": 7.95981823822498 + }, + { + "id": "10168", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10163", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10158", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10153", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10148", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10143", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10138", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10133", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10128", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "aurora quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "aurora quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10168", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10163", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10158", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10153", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10148", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10143", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10138", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10133", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10128", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "aurora-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "aurora rollout rollback purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "aurora rollout rollback purple", + "relevant": {}, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10168", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10163", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10158", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10153", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10148", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10143", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10138", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10133", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10128", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "cedar-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "cedar bakcup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "cedar bakcup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10465", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10460", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10455", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10450", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10445", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10440", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10435", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10430", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10425", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + } + ], + "corrections": [ + { + "from": "bakcup", + "to": [ + "backup" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "cedar back retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "cedar back retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10465", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10460", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10455", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10450", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10445", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10440", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10435", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10430", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10425", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"cedar backup retention\"", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"cedar backup retention\"", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "cedar backup from:700", + "relevant": { + "10303": 2 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "cedar backup from:700", + "relevant": { + "10303": 2 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "cedar backup chat:102", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "cedar backup chat:102", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10595", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10590", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10585", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10580", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10575", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10570", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10565", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10560", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10555", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "cedar backup date:2026-10-02", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10304", + "chatId": "102", + "match": null, + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "cedar backup after:2026-10-02 before:2026-10-03", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10304", + "chatId": "102", + "match": "words", + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "cedar AND backup AND retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "cedar backup OR retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "cedar backup OR retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "cedar snapshot lifetime", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "cedar snapshot lifetime", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": "anyWord", + "score": 9.865391899469309 + }, + { + "id": "11511", + "chatId": "106", + "match": "anyWord", + "score": 7.209540247771293 + }, + { + "id": "10465", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10460", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10455", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10450", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10445", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10440", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10435", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10430", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "cedar quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "cedar quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10465", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10460", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10455", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10450", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10445", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10440", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10435", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10430", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10425", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "cedar-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "cedar backup retention purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "cedar backup retention purple", + "relevant": {}, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10465", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10460", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10455", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10450", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10445", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10440", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10435", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10430", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10425", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "север-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 7.447603932068213 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 7.447603932068213 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "север сертифкат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "север-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "север сертифкат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "corrected", + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": "corrected", + "score": 7.447603932068213 + }, + { + "id": "10767", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10762", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10757", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10752", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10747", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10742", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10737", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "10732", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + } + ], + "corrections": [ + { + "from": "сертифкат", + "to": [ + "сертификат" + ] + } + ], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "север серти обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "север-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "север серти обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "beginnings", + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": "beginnings", + "score": 7.447603932068213 + }, + { + "id": "10767", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10762", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10757", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10752", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10747", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10742", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10737", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "10732", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"север сертификат обновление\"", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 2.8366598529202736 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "10862", + "chatId": "103", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "север-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"север сертификат обновление\"", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 2.8366598529202736 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "10727", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "север-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "север сертификат from:700", + "relevant": { + "10605": 2 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 5.213948397197031 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "север сертификат from:700", + "relevant": { + "10605": 2 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 5.213948397197031 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "север сертификат chat:103", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 5.213948397197031 + }, + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 5.059938729655221 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "север сертификат chat:103", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 5.213948397197031 + }, + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 5.059938729655221 + }, + { + "id": "10902", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10897", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10892", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10887", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10882", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10877", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10872", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "10867", + "chatId": "103", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "север сертификат date:2026-10-02", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 5.213948397197031 + }, + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 5.059938729655221 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "север сертификат after:2026-10-02 before:2026-10-03", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 5.213948397197031 + }, + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 5.059938729655221 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "север AND сертификат AND обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 7.447603932068213 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 7.447603932068213 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "север сертификат OR обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 7.447603932068213 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "север сертификат OR обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 7.447603932068213 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "север продление TLS", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "север-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "север продление TLS", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "anyWord", + "score": 2.5299693648276107 + }, + { + "id": "10767", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10762", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10757", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10752", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10747", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10742", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10737", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10732", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10727", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "север-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "север quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "север-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "север quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "anyWord", + "score": 2.5299693648276107 + }, + { + "id": "10767", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10762", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10757", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10752", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10747", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10742", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10737", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10732", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "10727", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "север-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "север сертификат обновление purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "север-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "север сертификат обновление purple", + "relevant": {}, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "anyWord", + "score": 7.589908094482832 + }, + { + "id": "10605", + "chatId": "103", + "match": "anyWord", + "score": 7.447603932068213 + }, + { + "id": "10767", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10762", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10757", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10752", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10747", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10742", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10737", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "10732", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "lumen-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "lumen invocie refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "lumen invocie refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11074", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11069", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11064", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11059", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11054", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11049", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11044", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11039", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11034", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + } + ], + "corrections": [ + { + "from": "invocie", + "to": [ + "invoice" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "lumen invo refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "lumen invo refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11074", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11069", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11064", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11059", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11054", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11049", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11044", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11039", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11034", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"lumen invoice refund\"", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"lumen invoice refund\"", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "lumen invoice from:700", + "relevant": { + "10907": 2 + }, + "hits": [ + { + "id": "10907", + "chatId": "104", + "match": null, + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "lumen invoice from:700", + "relevant": { + "10907": 2 + }, + "hits": [ + { + "id": "10907", + "chatId": "104", + "match": "words", + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "lumen invoice chat:104", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "lumen invoice chat:104", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11199", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11194", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11189", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11184", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11179", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11174", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11169", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11164", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11159", + "chatId": "104", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "lumen invoice date:2026-10-02", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "10908", + "chatId": "104", + "match": null, + "score": 4.340227569072966 + }, + { + "id": "10907", + "chatId": "104", + "match": null, + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "lumen invoice after:2026-10-02 before:2026-10-03", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "10908", + "chatId": "104", + "match": "words", + "score": 4.340227569072966 + }, + { + "id": "10907", + "chatId": "104", + "match": "words", + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "lumen AND invoice AND refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "lumen invoice OR refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "lumen invoice OR refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "lumen payment reimbursement", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "lumen payment reimbursement", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11074", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11069", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11064", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11059", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11054", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11049", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11044", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11039", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11034", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "lumen quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "lumen quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11074", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11069", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11064", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11059", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11054", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11049", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11044", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11039", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11034", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "lumen-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "lumen invoice refund purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "lumen invoice refund purple", + "relevant": {}, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11074", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11069", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11064", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11059", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11054", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11049", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11044", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11039", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11034", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "harbor-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "harbor relesae freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "harbor relesae freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11376", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11371", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11366", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11361", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11356", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11351", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11346", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11341", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11336", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + } + ], + "corrections": [ + { + "from": "relesae", + "to": [ + "release" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "harbor rele freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 8.213731853767285 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "harbor rele freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11376", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11371", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11366", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11361", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11356", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11351", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11346", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11341", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11336", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"harbor release freeze\"", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"harbor release freeze\"", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "harbor release from:700", + "relevant": { + "11209": 2 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": null, + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "harbor release from:700", + "relevant": { + "11209": 2 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "words", + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "harbor release chat:105", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "harbor release chat:105", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11501", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11496", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11491", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11486", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11481", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11476", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11471", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11466", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11461", + "chatId": "105", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "harbor release date:2026-10-02", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": null, + "score": 4.340227569072966 + }, + { + "id": "11210", + "chatId": "105", + "match": null, + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "harbor release after:2026-10-02 before:2026-10-03", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "words", + "score": 4.340227569072966 + }, + { + "id": "11210", + "chatId": "105", + "match": "words", + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "harbor AND release AND freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "harbor release OR freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "harbor release OR freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "harbor shipping pause", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "harbor shipping pause", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "anyWord", + "score": 10.129932022761462 + }, + { + "id": "11376", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11371", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11366", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11361", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11356", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11351", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11346", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11341", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11336", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [ + { + "from": "pause", + "to": [ + "passes" + ] + } + ], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "harbor quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "harbor quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11376", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11371", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11366", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11361", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11356", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11351", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11346", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11341", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11336", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "harbor-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "harbor release freeze purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "harbor release freeze purple", + "relevant": {}, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11376", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11371", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11366", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11361", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11356", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11351", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11346", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11341", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11336", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "maple-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "maple arhcive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "maple arhcive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11678", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11673", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11668", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11663", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11658", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11653", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11648", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11643", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + }, + { + "id": "11638", + "chatId": "999", + "match": "corrected", + "score": 6.696869489999996 + } + ], + "corrections": [ + { + "from": "arhcive", + "to": [ + "archive" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "maple arch restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "maple arch restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11678", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11673", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11668", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11663", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11658", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11653", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11648", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11643", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + }, + { + "id": "11638", + "chatId": "999", + "match": "beginnings", + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"maple archive restore\"", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"maple archive restore\"", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "maple archive from:700", + "relevant": { + "11511": 2 + }, + "hits": [ + { + "id": "11511", + "chatId": "106", + "match": null, + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "maple archive from:700", + "relevant": { + "11511": 2 + }, + "hits": [ + { + "id": "11511", + "chatId": "106", + "match": "words", + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "maple archive chat:106", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "maple archive chat:106", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11803", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11798", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11793", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11788", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11783", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11778", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11773", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11768", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "11763", + "chatId": "106", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "maple archive date:2026-10-02", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11512", + "chatId": "106", + "match": null, + "score": 4.220174468555267 + }, + { + "id": "11511", + "chatId": "106", + "match": null, + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "maple archive after:2026-10-02 before:2026-10-03", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11512", + "chatId": "106", + "match": "words", + "score": 4.220174468555267 + }, + { + "id": "11511", + "chatId": "106", + "match": "words", + "score": 4.220174468555267 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "maple AND archive AND restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "maple archive OR restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "maple archive OR restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "maple recover saved data", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "maple recover saved data", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11511", + "chatId": "106", + "match": "anyWord", + "score": 9.849732487596121 + }, + { + "id": "11209", + "chatId": "105", + "match": "anyWord", + "score": 7.95981823822498 + }, + { + "id": "11673", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11668", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11663", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11658", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11653", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11648", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11643", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11638", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "maple quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "maple quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11678", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11673", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11668", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11663", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11658", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11653", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11648", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11643", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11638", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "maple-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "maple archive restore purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "maple archive restore purple", + "relevant": {}, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11678", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11673", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11668", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11663", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11658", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11653", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11648", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11643", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + }, + { + "id": "11638", + "chatId": "999", + "match": "anyWord", + "score": 6.696869489999996 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "вектор-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 7.589908094482832 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 7.589908094482832 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "вектор миграця отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + }, + { + "id": "12065", + "chatId": "107", + "match": null, + "score": 8.959591949608743 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "вектор миграця отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "corrected", + "score": 7.589908094482832 + }, + { + "id": "11975", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11970", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11965", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11960", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11955", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11950", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11945", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11940", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "11935", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + } + ], + "corrections": [ + { + "from": "миграця", + "to": [ + "миграция" + ] + } + ], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "вектор мигр отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "вектор мигр отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "beginnings", + "score": 7.589908094482832 + }, + { + "id": "11975", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11970", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11965", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11960", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11955", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11950", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11945", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11940", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "11935", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"вектор миграция отмена\"", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 2.8366598529202736 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"вектор миграция отмена\"", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 2.8366598529202736 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "вектор миграция from:700", + "relevant": { + "11813": 2 + }, + "hits": [ + { + "id": "11813", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "вектор миграция from:700", + "relevant": { + "11813": 2 + }, + "hits": [ + { + "id": "11813", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "вектор миграция chat:107", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 5.059938729655221 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "вектор миграция chat:107", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 5.059938729655221 + }, + { + "id": "12110", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12105", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12100", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12095", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12090", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12085", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12080", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12075", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12070", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "вектор миграция date:2026-10-02", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 5.059938729655221 + }, + { + "id": "11813", + "chatId": "107", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "вектор миграция after:2026-10-02 before:2026-10-03", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 5.059938729655221 + }, + { + "id": "11813", + "chatId": "107", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "вектор AND миграция AND отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 7.589908094482832 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 7.589908094482832 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "вектор миграция OR отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 7.589908094482832 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "вектор миграция OR отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 7.589908094482832 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "вектор возврат прежней схемы", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "вектор возврат прежней схемы", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11813", + "chatId": "107", + "match": "anyWord", + "score": 10.426540254129044 + }, + { + "id": "11814", + "chatId": "107", + "match": "anyWord", + "score": 2.5299693648276107 + }, + { + "id": "11975", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11970", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11965", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11960", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11955", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11950", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11945", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11940", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [ + { + "from": "схемы", + "to": [ + "схему" + ] + } + ], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "вектор quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "вектор quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "anyWord", + "score": 2.5299693648276107 + }, + { + "id": "11975", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11970", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11965", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11960", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11955", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11950", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11945", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11940", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "11935", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "вектор-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "вектор миграция отмена purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "вектор миграция отмена purple", + "relevant": {}, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "anyWord", + "score": 7.589908094482832 + }, + { + "id": "11975", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11970", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11965", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11960", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11955", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11950", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11945", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11940", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "11935", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "prism-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "prism renweal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "prism renweal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12282", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12277", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12272", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12267", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12262", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12257", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12252", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12247", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + }, + { + "id": "12242", + "chatId": "999", + "match": "corrected", + "score": 6.700966604613545 + } + ], + "corrections": [ + { + "from": "renweal", + "to": [ + "renewal" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "prism rene discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "prism rene discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12282", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12277", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12272", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12267", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12262", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12257", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12252", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12247", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + }, + { + "id": "12242", + "chatId": "999", + "match": "beginnings", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"prism renewal discount\"", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"prism renewal discount\"", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 2.5044259701753258 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "prism renewal from:700", + "relevant": { + "12115": 2 + }, + "hits": [ + { + "id": "12115", + "chatId": "108", + "match": null, + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "prism renewal from:700", + "relevant": { + "12115": 2 + }, + "hits": [ + { + "id": "12115", + "chatId": "108", + "match": "words", + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "prism renewal chat:108", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "prism renewal chat:108", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12407", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12402", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12397", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12392", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12387", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12382", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12377", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12372", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + }, + { + "id": "12367", + "chatId": "108", + "match": "words", + "score": 4.4673110697423635 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "prism renewal date:2026-10-02", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12116", + "chatId": "108", + "match": null, + "score": 4.340227569072966 + }, + { + "id": "12115", + "chatId": "108", + "match": null, + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "prism renewal after:2026-10-02 before:2026-10-03", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12116", + "chatId": "108", + "match": "words", + "score": 4.340227569072966 + }, + { + "id": "12115", + "chatId": "108", + "match": "words", + "score": 4.340227569072966 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "prism AND renewal AND discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "prism renewal OR discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "prism renewal OR discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "prism subscription price reduction", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "prism subscription price reduction", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12282", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12277", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12272", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12267", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12262", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12257", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12252", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12247", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + }, + { + "id": "12242", + "chatId": "999", + "match": "anyWord", + "score": 4.4673110697423635 + } + ], + "corrections": [ + { + "from": "price", + "to": [ + "prism" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "prism quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "prism quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12282", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12277", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12272", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12267", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12262", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12257", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12252", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12247", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + }, + { + "id": "12242", + "chatId": "999", + "match": "anyWord", + "score": 2.2336555348711817 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "prism-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "prism renewal discount purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "prism renewal discount purple", + "relevant": {}, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12282", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12277", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12272", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12267", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12262", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12257", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12252", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12247", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + }, + { + "id": "12242", + "chatId": "999", + "match": "anyWord", + "score": 6.700966604613545 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + } + ] +} diff --git a/bench/message-search-quality/corpus.json b/bench/message-search-quality/corpus.json new file mode 100644 index 000000000..fe361f1ab --- /dev/null +++ b/bench/message-search-quality/corpus.json @@ -0,0 +1,19452 @@ +{ + "chats": [ + { + "id": "101", + "title": "Synthetic aurora decisions" + }, + { + "id": "102", + "title": "Synthetic cedar decisions" + }, + { + "id": "103", + "title": "Synthetic север decisions" + }, + { + "id": "104", + "title": "Synthetic lumen decisions" + }, + { + "id": "105", + "title": "Synthetic harbor decisions" + }, + { + "id": "106", + "title": "Synthetic maple decisions" + }, + { + "id": "107", + "title": "Synthetic вектор decisions" + }, + { + "id": "108", + "title": "Synthetic prism decisions" + }, + { + "id": "999", + "title": "Synthetic busy operations" + } + ], + "messages": [ + { + "id": "10001", + "chatId": "101", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Aurora rollout rollback decision: disable the canary and restore build 17." + }, + { + "id": "10002", + "chatId": "101", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "The Aurora rollout rollback owner is Mira; the decision is recorded above." + }, + { + "id": "10003", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 0." + }, + { + "id": "10004", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 1." + }, + { + "id": "10005", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 2." + }, + { + "id": "10006", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 3." + }, + { + "id": "10007", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "10008", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 5." + }, + { + "id": "10009", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 6." + }, + { + "id": "10010", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 7." + }, + { + "id": "10011", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 8." + }, + { + "id": "10012", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "10013", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 10." + }, + { + "id": "10014", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 11." + }, + { + "id": "10015", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 12." + }, + { + "id": "10016", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 13." + }, + { + "id": "10017", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "10018", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 15." + }, + { + "id": "10019", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 16." + }, + { + "id": "10020", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 17." + }, + { + "id": "10021", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 18." + }, + { + "id": "10022", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "10023", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 20." + }, + { + "id": "10024", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 21." + }, + { + "id": "10025", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 22." + }, + { + "id": "10026", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 23." + }, + { + "id": "10027", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "10028", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 25." + }, + { + "id": "10029", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 26." + }, + { + "id": "10030", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 27." + }, + { + "id": "10031", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 28." + }, + { + "id": "10032", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "10033", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 30." + }, + { + "id": "10034", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 31." + }, + { + "id": "10035", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 32." + }, + { + "id": "10036", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 33." + }, + { + "id": "10037", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "10038", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 35." + }, + { + "id": "10039", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 36." + }, + { + "id": "10040", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 37." + }, + { + "id": "10041", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 38." + }, + { + "id": "10042", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "10043", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 40." + }, + { + "id": "10044", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 41." + }, + { + "id": "10045", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 42." + }, + { + "id": "10046", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 43." + }, + { + "id": "10047", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "10048", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 45." + }, + { + "id": "10049", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 46." + }, + { + "id": "10050", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 47." + }, + { + "id": "10051", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 48." + }, + { + "id": "10052", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "10053", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 50." + }, + { + "id": "10054", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 51." + }, + { + "id": "10055", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 52." + }, + { + "id": "10056", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 53." + }, + { + "id": "10057", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "10058", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 55." + }, + { + "id": "10059", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 56." + }, + { + "id": "10060", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 57." + }, + { + "id": "10061", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 58." + }, + { + "id": "10062", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "10063", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 60." + }, + { + "id": "10064", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 61." + }, + { + "id": "10065", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 62." + }, + { + "id": "10066", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 63." + }, + { + "id": "10067", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "10068", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 65." + }, + { + "id": "10069", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 66." + }, + { + "id": "10070", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 67." + }, + { + "id": "10071", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 68." + }, + { + "id": "10072", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "10073", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 70." + }, + { + "id": "10074", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 71." + }, + { + "id": "10075", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 72." + }, + { + "id": "10076", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 73." + }, + { + "id": "10077", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "10078", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 75." + }, + { + "id": "10079", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 76." + }, + { + "id": "10080", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 77." + }, + { + "id": "10081", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 78." + }, + { + "id": "10082", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "10083", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 80." + }, + { + "id": "10084", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 81." + }, + { + "id": "10085", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 82." + }, + { + "id": "10086", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 83." + }, + { + "id": "10087", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "10088", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 85." + }, + { + "id": "10089", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 86." + }, + { + "id": "10090", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 87." + }, + { + "id": "10091", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 88." + }, + { + "id": "10092", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "10093", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 90." + }, + { + "id": "10094", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 91." + }, + { + "id": "10095", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 92." + }, + { + "id": "10096", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 93." + }, + { + "id": "10097", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "10098", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 95." + }, + { + "id": "10099", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 96." + }, + { + "id": "10100", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 97." + }, + { + "id": "10101", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 98." + }, + { + "id": "10102", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "10103", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 100." + }, + { + "id": "10104", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 101." + }, + { + "id": "10105", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 102." + }, + { + "id": "10106", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 103." + }, + { + "id": "10107", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "10108", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 105." + }, + { + "id": "10109", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 106." + }, + { + "id": "10110", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 107." + }, + { + "id": "10111", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 108." + }, + { + "id": "10112", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "10113", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 110." + }, + { + "id": "10114", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 111." + }, + { + "id": "10115", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 112." + }, + { + "id": "10116", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 113." + }, + { + "id": "10117", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "10118", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 115." + }, + { + "id": "10119", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 116." + }, + { + "id": "10120", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 117." + }, + { + "id": "10121", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 118." + }, + { + "id": "10122", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "10123", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 120." + }, + { + "id": "10124", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 121." + }, + { + "id": "10125", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 122." + }, + { + "id": "10126", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 123." + }, + { + "id": "10127", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "10128", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 125." + }, + { + "id": "10129", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 126." + }, + { + "id": "10130", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 127." + }, + { + "id": "10131", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 128." + }, + { + "id": "10132", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "10133", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 130." + }, + { + "id": "10134", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 131." + }, + { + "id": "10135", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 132." + }, + { + "id": "10136", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 133." + }, + { + "id": "10137", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "10138", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 135." + }, + { + "id": "10139", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 136." + }, + { + "id": "10140", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 137." + }, + { + "id": "10141", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 138." + }, + { + "id": "10142", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "10143", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 140." + }, + { + "id": "10144", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 141." + }, + { + "id": "10145", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 142." + }, + { + "id": "10146", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 143." + }, + { + "id": "10147", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "10148", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 145." + }, + { + "id": "10149", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 146." + }, + { + "id": "10150", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 147." + }, + { + "id": "10151", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 148." + }, + { + "id": "10152", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "10153", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 150." + }, + { + "id": "10154", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 151." + }, + { + "id": "10155", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 152." + }, + { + "id": "10156", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 153." + }, + { + "id": "10157", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "10158", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 155." + }, + { + "id": "10159", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 156." + }, + { + "id": "10160", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 157." + }, + { + "id": "10161", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 158." + }, + { + "id": "10162", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "10163", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 160." + }, + { + "id": "10164", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 161." + }, + { + "id": "10165", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 162." + }, + { + "id": "10166", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 163." + }, + { + "id": "10167", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "10168", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 165." + }, + { + "id": "10169", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 166." + }, + { + "id": "10170", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 167." + }, + { + "id": "10171", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 168." + }, + { + "id": "10172", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "10173", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 170." + }, + { + "id": "10174", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 171." + }, + { + "id": "10175", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 172." + }, + { + "id": "10176", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 173." + }, + { + "id": "10177", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "10178", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 175." + }, + { + "id": "10179", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 176." + }, + { + "id": "10180", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 177." + }, + { + "id": "10181", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 178." + }, + { + "id": "10182", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "10183", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 180." + }, + { + "id": "10184", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 181." + }, + { + "id": "10185", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 182." + }, + { + "id": "10186", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 183." + }, + { + "id": "10187", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "10188", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 185." + }, + { + "id": "10189", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 186." + }, + { + "id": "10190", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 187." + }, + { + "id": "10191", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 188." + }, + { + "id": "10192", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "10193", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 190." + }, + { + "id": "10194", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 191." + }, + { + "id": "10195", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 192." + }, + { + "id": "10196", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 193." + }, + { + "id": "10197", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "10198", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 195." + }, + { + "id": "10199", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 196." + }, + { + "id": "10200", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 197." + }, + { + "id": "10201", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 198." + }, + { + "id": "10202", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "10203", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 200." + }, + { + "id": "10204", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 201." + }, + { + "id": "10205", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 202." + }, + { + "id": "10206", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 203." + }, + { + "id": "10207", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "10208", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 205." + }, + { + "id": "10209", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 206." + }, + { + "id": "10210", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 207." + }, + { + "id": "10211", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 208." + }, + { + "id": "10212", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "10213", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 210." + }, + { + "id": "10214", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 211." + }, + { + "id": "10215", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 212." + }, + { + "id": "10216", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 213." + }, + { + "id": "10217", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "10218", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 215." + }, + { + "id": "10219", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 216." + }, + { + "id": "10220", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 217." + }, + { + "id": "10221", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 218." + }, + { + "id": "10222", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "10223", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 220." + }, + { + "id": "10224", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 221." + }, + { + "id": "10225", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 222." + }, + { + "id": "10226", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 223." + }, + { + "id": "10227", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "10228", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 225." + }, + { + "id": "10229", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 226." + }, + { + "id": "10230", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 227." + }, + { + "id": "10231", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 228." + }, + { + "id": "10232", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "10233", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 230." + }, + { + "id": "10234", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 231." + }, + { + "id": "10235", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 232." + }, + { + "id": "10236", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 233." + }, + { + "id": "10237", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "10238", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 235." + }, + { + "id": "10239", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 236." + }, + { + "id": "10240", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 237." + }, + { + "id": "10241", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 238." + }, + { + "id": "10242", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "10243", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 240." + }, + { + "id": "10244", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 241." + }, + { + "id": "10245", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 242." + }, + { + "id": "10246", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 243." + }, + { + "id": "10247", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "10248", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 245." + }, + { + "id": "10249", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 246." + }, + { + "id": "10250", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 247." + }, + { + "id": "10251", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 248." + }, + { + "id": "10252", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "10253", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 250." + }, + { + "id": "10254", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 251." + }, + { + "id": "10255", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 252." + }, + { + "id": "10256", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 253." + }, + { + "id": "10257", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "10258", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 255." + }, + { + "id": "10259", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 256." + }, + { + "id": "10260", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 257." + }, + { + "id": "10261", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 258." + }, + { + "id": "10262", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "10263", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 260." + }, + { + "id": "10264", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 261." + }, + { + "id": "10265", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 262." + }, + { + "id": "10266", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 263." + }, + { + "id": "10267", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "10268", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 265." + }, + { + "id": "10269", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 266." + }, + { + "id": "10270", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 267." + }, + { + "id": "10271", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 268." + }, + { + "id": "10272", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "10273", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 270." + }, + { + "id": "10274", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 271." + }, + { + "id": "10275", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 272." + }, + { + "id": "10276", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 273." + }, + { + "id": "10277", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "10278", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 275." + }, + { + "id": "10279", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 276." + }, + { + "id": "10280", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 277." + }, + { + "id": "10281", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 278." + }, + { + "id": "10282", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "10283", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 280." + }, + { + "id": "10284", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 281." + }, + { + "id": "10285", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 282." + }, + { + "id": "10286", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 283." + }, + { + "id": "10287", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "10288", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 285." + }, + { + "id": "10289", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 286." + }, + { + "id": "10290", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 287." + }, + { + "id": "10291", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 288." + }, + { + "id": "10292", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "10293", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 290." + }, + { + "id": "10294", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 291." + }, + { + "id": "10295", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 292." + }, + { + "id": "10296", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 293." + }, + { + "id": "10297", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "10298", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback: agenda only; no decision yet. Thread 295." + }, + { + "id": "10299", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora rollout rollback? Please add this question to the meeting. Thread 296." + }, + { + "id": "10300", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 297." + }, + { + "id": "10301", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 298." + }, + { + "id": "10302", + "chatId": "101", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "10303", + "chatId": "102", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Cedar backup retention decision: keep daily snapshots for 35 days." + }, + { + "id": "10304", + "chatId": "102", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "The Cedar backup retention owner is Niko; the policy is recorded above." + }, + { + "id": "10305", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 0." + }, + { + "id": "10306", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 1." + }, + { + "id": "10307", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 2." + }, + { + "id": "10308", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 3." + }, + { + "id": "10309", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "10310", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 5." + }, + { + "id": "10311", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 6." + }, + { + "id": "10312", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 7." + }, + { + "id": "10313", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 8." + }, + { + "id": "10314", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "10315", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 10." + }, + { + "id": "10316", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 11." + }, + { + "id": "10317", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 12." + }, + { + "id": "10318", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 13." + }, + { + "id": "10319", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "10320", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 15." + }, + { + "id": "10321", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 16." + }, + { + "id": "10322", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 17." + }, + { + "id": "10323", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 18." + }, + { + "id": "10324", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "10325", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 20." + }, + { + "id": "10326", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 21." + }, + { + "id": "10327", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 22." + }, + { + "id": "10328", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 23." + }, + { + "id": "10329", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "10330", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 25." + }, + { + "id": "10331", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 26." + }, + { + "id": "10332", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 27." + }, + { + "id": "10333", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 28." + }, + { + "id": "10334", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "10335", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 30." + }, + { + "id": "10336", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 31." + }, + { + "id": "10337", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 32." + }, + { + "id": "10338", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 33." + }, + { + "id": "10339", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "10340", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 35." + }, + { + "id": "10341", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 36." + }, + { + "id": "10342", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 37." + }, + { + "id": "10343", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 38." + }, + { + "id": "10344", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "10345", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 40." + }, + { + "id": "10346", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 41." + }, + { + "id": "10347", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 42." + }, + { + "id": "10348", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 43." + }, + { + "id": "10349", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "10350", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 45." + }, + { + "id": "10351", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 46." + }, + { + "id": "10352", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 47." + }, + { + "id": "10353", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 48." + }, + { + "id": "10354", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "10355", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 50." + }, + { + "id": "10356", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 51." + }, + { + "id": "10357", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 52." + }, + { + "id": "10358", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 53." + }, + { + "id": "10359", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "10360", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 55." + }, + { + "id": "10361", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 56." + }, + { + "id": "10362", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 57." + }, + { + "id": "10363", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 58." + }, + { + "id": "10364", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "10365", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 60." + }, + { + "id": "10366", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 61." + }, + { + "id": "10367", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 62." + }, + { + "id": "10368", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 63." + }, + { + "id": "10369", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "10370", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 65." + }, + { + "id": "10371", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 66." + }, + { + "id": "10372", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 67." + }, + { + "id": "10373", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 68." + }, + { + "id": "10374", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "10375", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 70." + }, + { + "id": "10376", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 71." + }, + { + "id": "10377", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 72." + }, + { + "id": "10378", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 73." + }, + { + "id": "10379", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "10380", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 75." + }, + { + "id": "10381", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 76." + }, + { + "id": "10382", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 77." + }, + { + "id": "10383", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 78." + }, + { + "id": "10384", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "10385", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 80." + }, + { + "id": "10386", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 81." + }, + { + "id": "10387", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 82." + }, + { + "id": "10388", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 83." + }, + { + "id": "10389", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "10390", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 85." + }, + { + "id": "10391", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 86." + }, + { + "id": "10392", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 87." + }, + { + "id": "10393", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 88." + }, + { + "id": "10394", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "10395", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 90." + }, + { + "id": "10396", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 91." + }, + { + "id": "10397", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 92." + }, + { + "id": "10398", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 93." + }, + { + "id": "10399", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "10400", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 95." + }, + { + "id": "10401", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 96." + }, + { + "id": "10402", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 97." + }, + { + "id": "10403", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 98." + }, + { + "id": "10404", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "10405", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 100." + }, + { + "id": "10406", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 101." + }, + { + "id": "10407", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 102." + }, + { + "id": "10408", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 103." + }, + { + "id": "10409", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "10410", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 105." + }, + { + "id": "10411", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 106." + }, + { + "id": "10412", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 107." + }, + { + "id": "10413", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 108." + }, + { + "id": "10414", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "10415", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 110." + }, + { + "id": "10416", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 111." + }, + { + "id": "10417", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 112." + }, + { + "id": "10418", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 113." + }, + { + "id": "10419", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "10420", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 115." + }, + { + "id": "10421", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 116." + }, + { + "id": "10422", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 117." + }, + { + "id": "10423", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 118." + }, + { + "id": "10424", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "10425", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 120." + }, + { + "id": "10426", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 121." + }, + { + "id": "10427", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 122." + }, + { + "id": "10428", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 123." + }, + { + "id": "10429", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "10430", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 125." + }, + { + "id": "10431", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 126." + }, + { + "id": "10432", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 127." + }, + { + "id": "10433", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 128." + }, + { + "id": "10434", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "10435", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 130." + }, + { + "id": "10436", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 131." + }, + { + "id": "10437", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 132." + }, + { + "id": "10438", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 133." + }, + { + "id": "10439", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "10440", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 135." + }, + { + "id": "10441", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 136." + }, + { + "id": "10442", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 137." + }, + { + "id": "10443", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 138." + }, + { + "id": "10444", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "10445", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 140." + }, + { + "id": "10446", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 141." + }, + { + "id": "10447", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 142." + }, + { + "id": "10448", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 143." + }, + { + "id": "10449", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "10450", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 145." + }, + { + "id": "10451", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 146." + }, + { + "id": "10452", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 147." + }, + { + "id": "10453", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 148." + }, + { + "id": "10454", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "10455", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 150." + }, + { + "id": "10456", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 151." + }, + { + "id": "10457", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 152." + }, + { + "id": "10458", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 153." + }, + { + "id": "10459", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "10460", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 155." + }, + { + "id": "10461", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 156." + }, + { + "id": "10462", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 157." + }, + { + "id": "10463", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 158." + }, + { + "id": "10464", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "10465", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 160." + }, + { + "id": "10466", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 161." + }, + { + "id": "10467", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 162." + }, + { + "id": "10468", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 163." + }, + { + "id": "10469", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "10470", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 165." + }, + { + "id": "10471", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 166." + }, + { + "id": "10472", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 167." + }, + { + "id": "10473", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 168." + }, + { + "id": "10474", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "10475", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 170." + }, + { + "id": "10476", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 171." + }, + { + "id": "10477", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 172." + }, + { + "id": "10478", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 173." + }, + { + "id": "10479", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "10480", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 175." + }, + { + "id": "10481", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 176." + }, + { + "id": "10482", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 177." + }, + { + "id": "10483", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 178." + }, + { + "id": "10484", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "10485", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 180." + }, + { + "id": "10486", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 181." + }, + { + "id": "10487", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 182." + }, + { + "id": "10488", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 183." + }, + { + "id": "10489", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "10490", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 185." + }, + { + "id": "10491", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 186." + }, + { + "id": "10492", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 187." + }, + { + "id": "10493", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 188." + }, + { + "id": "10494", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "10495", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 190." + }, + { + "id": "10496", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 191." + }, + { + "id": "10497", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 192." + }, + { + "id": "10498", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 193." + }, + { + "id": "10499", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "10500", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 195." + }, + { + "id": "10501", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 196." + }, + { + "id": "10502", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 197." + }, + { + "id": "10503", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 198." + }, + { + "id": "10504", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "10505", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 200." + }, + { + "id": "10506", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 201." + }, + { + "id": "10507", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 202." + }, + { + "id": "10508", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 203." + }, + { + "id": "10509", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "10510", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 205." + }, + { + "id": "10511", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 206." + }, + { + "id": "10512", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 207." + }, + { + "id": "10513", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 208." + }, + { + "id": "10514", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "10515", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 210." + }, + { + "id": "10516", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 211." + }, + { + "id": "10517", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 212." + }, + { + "id": "10518", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 213." + }, + { + "id": "10519", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "10520", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 215." + }, + { + "id": "10521", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 216." + }, + { + "id": "10522", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 217." + }, + { + "id": "10523", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 218." + }, + { + "id": "10524", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "10525", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 220." + }, + { + "id": "10526", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 221." + }, + { + "id": "10527", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 222." + }, + { + "id": "10528", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 223." + }, + { + "id": "10529", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "10530", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 225." + }, + { + "id": "10531", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 226." + }, + { + "id": "10532", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 227." + }, + { + "id": "10533", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 228." + }, + { + "id": "10534", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "10535", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 230." + }, + { + "id": "10536", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 231." + }, + { + "id": "10537", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 232." + }, + { + "id": "10538", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 233." + }, + { + "id": "10539", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "10540", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 235." + }, + { + "id": "10541", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 236." + }, + { + "id": "10542", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 237." + }, + { + "id": "10543", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 238." + }, + { + "id": "10544", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "10545", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 240." + }, + { + "id": "10546", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 241." + }, + { + "id": "10547", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 242." + }, + { + "id": "10548", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 243." + }, + { + "id": "10549", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "10550", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 245." + }, + { + "id": "10551", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 246." + }, + { + "id": "10552", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 247." + }, + { + "id": "10553", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 248." + }, + { + "id": "10554", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "10555", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 250." + }, + { + "id": "10556", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 251." + }, + { + "id": "10557", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 252." + }, + { + "id": "10558", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 253." + }, + { + "id": "10559", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "10560", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 255." + }, + { + "id": "10561", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 256." + }, + { + "id": "10562", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 257." + }, + { + "id": "10563", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 258." + }, + { + "id": "10564", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "10565", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 260." + }, + { + "id": "10566", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 261." + }, + { + "id": "10567", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 262." + }, + { + "id": "10568", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 263." + }, + { + "id": "10569", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "10570", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 265." + }, + { + "id": "10571", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 266." + }, + { + "id": "10572", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 267." + }, + { + "id": "10573", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 268." + }, + { + "id": "10574", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "10575", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 270." + }, + { + "id": "10576", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 271." + }, + { + "id": "10577", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 272." + }, + { + "id": "10578", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 273." + }, + { + "id": "10579", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "10580", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 275." + }, + { + "id": "10581", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 276." + }, + { + "id": "10582", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 277." + }, + { + "id": "10583", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 278." + }, + { + "id": "10584", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "10585", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 280." + }, + { + "id": "10586", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 281." + }, + { + "id": "10587", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 282." + }, + { + "id": "10588", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 283." + }, + { + "id": "10589", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "10590", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 285." + }, + { + "id": "10591", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 286." + }, + { + "id": "10592", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 287." + }, + { + "id": "10593", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 288." + }, + { + "id": "10594", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "10595", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 290." + }, + { + "id": "10596", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 291." + }, + { + "id": "10597", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 292." + }, + { + "id": "10598", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 293." + }, + { + "id": "10599", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "10600", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention: agenda only; no decision yet. Thread 295." + }, + { + "id": "10601", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar backup retention? Please add this question to the meeting. Thread 296." + }, + { + "id": "10602", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 297." + }, + { + "id": "10603", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 298." + }, + { + "id": "10604", + "chatId": "102", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "10605", + "chatId": "103", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря." + }, + { + "id": "10606", + "chatId": "103", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Север сертификат обновление проверено, владелец Ира." + }, + { + "id": "10607", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 0." + }, + { + "id": "10608", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 1." + }, + { + "id": "10609", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 2." + }, + { + "id": "10610", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 3." + }, + { + "id": "10611", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "10612", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 5." + }, + { + "id": "10613", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 6." + }, + { + "id": "10614", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 7." + }, + { + "id": "10615", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 8." + }, + { + "id": "10616", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "10617", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 10." + }, + { + "id": "10618", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 11." + }, + { + "id": "10619", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 12." + }, + { + "id": "10620", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 13." + }, + { + "id": "10621", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "10622", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 15." + }, + { + "id": "10623", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 16." + }, + { + "id": "10624", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 17." + }, + { + "id": "10625", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 18." + }, + { + "id": "10626", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "10627", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 20." + }, + { + "id": "10628", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 21." + }, + { + "id": "10629", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 22." + }, + { + "id": "10630", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 23." + }, + { + "id": "10631", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "10632", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 25." + }, + { + "id": "10633", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 26." + }, + { + "id": "10634", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 27." + }, + { + "id": "10635", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 28." + }, + { + "id": "10636", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "10637", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 30." + }, + { + "id": "10638", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 31." + }, + { + "id": "10639", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 32." + }, + { + "id": "10640", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 33." + }, + { + "id": "10641", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "10642", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 35." + }, + { + "id": "10643", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 36." + }, + { + "id": "10644", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 37." + }, + { + "id": "10645", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 38." + }, + { + "id": "10646", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "10647", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 40." + }, + { + "id": "10648", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 41." + }, + { + "id": "10649", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 42." + }, + { + "id": "10650", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 43." + }, + { + "id": "10651", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "10652", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 45." + }, + { + "id": "10653", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 46." + }, + { + "id": "10654", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 47." + }, + { + "id": "10655", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 48." + }, + { + "id": "10656", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "10657", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 50." + }, + { + "id": "10658", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 51." + }, + { + "id": "10659", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 52." + }, + { + "id": "10660", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 53." + }, + { + "id": "10661", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "10662", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 55." + }, + { + "id": "10663", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 56." + }, + { + "id": "10664", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 57." + }, + { + "id": "10665", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 58." + }, + { + "id": "10666", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "10667", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 60." + }, + { + "id": "10668", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 61." + }, + { + "id": "10669", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 62." + }, + { + "id": "10670", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 63." + }, + { + "id": "10671", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "10672", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 65." + }, + { + "id": "10673", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 66." + }, + { + "id": "10674", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 67." + }, + { + "id": "10675", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 68." + }, + { + "id": "10676", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "10677", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 70." + }, + { + "id": "10678", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 71." + }, + { + "id": "10679", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 72." + }, + { + "id": "10680", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 73." + }, + { + "id": "10681", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "10682", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 75." + }, + { + "id": "10683", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 76." + }, + { + "id": "10684", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 77." + }, + { + "id": "10685", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 78." + }, + { + "id": "10686", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "10687", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 80." + }, + { + "id": "10688", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 81." + }, + { + "id": "10689", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 82." + }, + { + "id": "10690", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 83." + }, + { + "id": "10691", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "10692", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 85." + }, + { + "id": "10693", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 86." + }, + { + "id": "10694", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 87." + }, + { + "id": "10695", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 88." + }, + { + "id": "10696", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "10697", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 90." + }, + { + "id": "10698", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 91." + }, + { + "id": "10699", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 92." + }, + { + "id": "10700", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 93." + }, + { + "id": "10701", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "10702", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 95." + }, + { + "id": "10703", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 96." + }, + { + "id": "10704", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 97." + }, + { + "id": "10705", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 98." + }, + { + "id": "10706", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "10707", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 100." + }, + { + "id": "10708", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 101." + }, + { + "id": "10709", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 102." + }, + { + "id": "10710", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 103." + }, + { + "id": "10711", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "10712", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 105." + }, + { + "id": "10713", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 106." + }, + { + "id": "10714", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 107." + }, + { + "id": "10715", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 108." + }, + { + "id": "10716", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "10717", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 110." + }, + { + "id": "10718", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 111." + }, + { + "id": "10719", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 112." + }, + { + "id": "10720", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 113." + }, + { + "id": "10721", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "10722", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 115." + }, + { + "id": "10723", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 116." + }, + { + "id": "10724", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 117." + }, + { + "id": "10725", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 118." + }, + { + "id": "10726", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "10727", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 120." + }, + { + "id": "10728", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 121." + }, + { + "id": "10729", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 122." + }, + { + "id": "10730", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 123." + }, + { + "id": "10731", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "10732", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 125." + }, + { + "id": "10733", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 126." + }, + { + "id": "10734", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 127." + }, + { + "id": "10735", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 128." + }, + { + "id": "10736", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "10737", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 130." + }, + { + "id": "10738", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 131." + }, + { + "id": "10739", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 132." + }, + { + "id": "10740", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 133." + }, + { + "id": "10741", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "10742", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 135." + }, + { + "id": "10743", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 136." + }, + { + "id": "10744", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 137." + }, + { + "id": "10745", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 138." + }, + { + "id": "10746", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "10747", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 140." + }, + { + "id": "10748", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 141." + }, + { + "id": "10749", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 142." + }, + { + "id": "10750", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 143." + }, + { + "id": "10751", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "10752", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 145." + }, + { + "id": "10753", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 146." + }, + { + "id": "10754", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 147." + }, + { + "id": "10755", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 148." + }, + { + "id": "10756", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "10757", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 150." + }, + { + "id": "10758", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 151." + }, + { + "id": "10759", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 152." + }, + { + "id": "10760", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 153." + }, + { + "id": "10761", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "10762", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 155." + }, + { + "id": "10763", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 156." + }, + { + "id": "10764", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 157." + }, + { + "id": "10765", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 158." + }, + { + "id": "10766", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "10767", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 160." + }, + { + "id": "10768", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 161." + }, + { + "id": "10769", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 162." + }, + { + "id": "10770", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 163." + }, + { + "id": "10771", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "10772", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 165." + }, + { + "id": "10773", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 166." + }, + { + "id": "10774", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 167." + }, + { + "id": "10775", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 168." + }, + { + "id": "10776", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "10777", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 170." + }, + { + "id": "10778", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 171." + }, + { + "id": "10779", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 172." + }, + { + "id": "10780", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 173." + }, + { + "id": "10781", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "10782", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 175." + }, + { + "id": "10783", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 176." + }, + { + "id": "10784", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 177." + }, + { + "id": "10785", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 178." + }, + { + "id": "10786", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "10787", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 180." + }, + { + "id": "10788", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 181." + }, + { + "id": "10789", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 182." + }, + { + "id": "10790", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 183." + }, + { + "id": "10791", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "10792", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 185." + }, + { + "id": "10793", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 186." + }, + { + "id": "10794", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 187." + }, + { + "id": "10795", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 188." + }, + { + "id": "10796", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "10797", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 190." + }, + { + "id": "10798", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 191." + }, + { + "id": "10799", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 192." + }, + { + "id": "10800", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 193." + }, + { + "id": "10801", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "10802", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 195." + }, + { + "id": "10803", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 196." + }, + { + "id": "10804", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 197." + }, + { + "id": "10805", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 198." + }, + { + "id": "10806", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "10807", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 200." + }, + { + "id": "10808", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 201." + }, + { + "id": "10809", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 202." + }, + { + "id": "10810", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 203." + }, + { + "id": "10811", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "10812", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 205." + }, + { + "id": "10813", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 206." + }, + { + "id": "10814", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 207." + }, + { + "id": "10815", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 208." + }, + { + "id": "10816", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "10817", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 210." + }, + { + "id": "10818", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 211." + }, + { + "id": "10819", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 212." + }, + { + "id": "10820", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 213." + }, + { + "id": "10821", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "10822", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 215." + }, + { + "id": "10823", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 216." + }, + { + "id": "10824", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 217." + }, + { + "id": "10825", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 218." + }, + { + "id": "10826", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "10827", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 220." + }, + { + "id": "10828", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 221." + }, + { + "id": "10829", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 222." + }, + { + "id": "10830", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 223." + }, + { + "id": "10831", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "10832", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 225." + }, + { + "id": "10833", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 226." + }, + { + "id": "10834", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 227." + }, + { + "id": "10835", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 228." + }, + { + "id": "10836", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "10837", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 230." + }, + { + "id": "10838", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 231." + }, + { + "id": "10839", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 232." + }, + { + "id": "10840", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 233." + }, + { + "id": "10841", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "10842", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 235." + }, + { + "id": "10843", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 236." + }, + { + "id": "10844", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 237." + }, + { + "id": "10845", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 238." + }, + { + "id": "10846", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "10847", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 240." + }, + { + "id": "10848", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 241." + }, + { + "id": "10849", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 242." + }, + { + "id": "10850", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 243." + }, + { + "id": "10851", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "10852", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 245." + }, + { + "id": "10853", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 246." + }, + { + "id": "10854", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 247." + }, + { + "id": "10855", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 248." + }, + { + "id": "10856", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "10857", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 250." + }, + { + "id": "10858", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 251." + }, + { + "id": "10859", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 252." + }, + { + "id": "10860", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 253." + }, + { + "id": "10861", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "10862", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 255." + }, + { + "id": "10863", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 256." + }, + { + "id": "10864", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 257." + }, + { + "id": "10865", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 258." + }, + { + "id": "10866", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "10867", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 260." + }, + { + "id": "10868", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 261." + }, + { + "id": "10869", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 262." + }, + { + "id": "10870", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 263." + }, + { + "id": "10871", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "10872", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 265." + }, + { + "id": "10873", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 266." + }, + { + "id": "10874", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 267." + }, + { + "id": "10875", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 268." + }, + { + "id": "10876", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "10877", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 270." + }, + { + "id": "10878", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 271." + }, + { + "id": "10879", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 272." + }, + { + "id": "10880", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 273." + }, + { + "id": "10881", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "10882", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 275." + }, + { + "id": "10883", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 276." + }, + { + "id": "10884", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 277." + }, + { + "id": "10885", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 278." + }, + { + "id": "10886", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "10887", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 280." + }, + { + "id": "10888", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 281." + }, + { + "id": "10889", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 282." + }, + { + "id": "10890", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 283." + }, + { + "id": "10891", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "10892", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 285." + }, + { + "id": "10893", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 286." + }, + { + "id": "10894", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 287." + }, + { + "id": "10895", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 288." + }, + { + "id": "10896", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "10897", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 290." + }, + { + "id": "10898", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 291." + }, + { + "id": "10899", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 292." + }, + { + "id": "10900", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 293." + }, + { + "id": "10901", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "10902", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление: agenda only; no decision yet. Thread 295." + }, + { + "id": "10903", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север сертификат обновление? Please add this question to the meeting. Thread 296." + }, + { + "id": "10904", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 297." + }, + { + "id": "10905", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 298." + }, + { + "id": "10906", + "chatId": "103", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "10907", + "chatId": "104", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Lumen invoice refund approved: return 480 credits to the original payer." + }, + { + "id": "10908", + "chatId": "104", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "The Lumen invoice refund owner is Eva; approval is recorded above." + }, + { + "id": "10909", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 0." + }, + { + "id": "10910", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 1." + }, + { + "id": "10911", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 2." + }, + { + "id": "10912", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 3." + }, + { + "id": "10913", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "10914", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 5." + }, + { + "id": "10915", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 6." + }, + { + "id": "10916", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 7." + }, + { + "id": "10917", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 8." + }, + { + "id": "10918", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "10919", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 10." + }, + { + "id": "10920", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 11." + }, + { + "id": "10921", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 12." + }, + { + "id": "10922", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 13." + }, + { + "id": "10923", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "10924", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 15." + }, + { + "id": "10925", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 16." + }, + { + "id": "10926", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 17." + }, + { + "id": "10927", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 18." + }, + { + "id": "10928", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "10929", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 20." + }, + { + "id": "10930", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 21." + }, + { + "id": "10931", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 22." + }, + { + "id": "10932", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 23." + }, + { + "id": "10933", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "10934", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 25." + }, + { + "id": "10935", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 26." + }, + { + "id": "10936", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 27." + }, + { + "id": "10937", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 28." + }, + { + "id": "10938", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "10939", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 30." + }, + { + "id": "10940", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 31." + }, + { + "id": "10941", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 32." + }, + { + "id": "10942", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 33." + }, + { + "id": "10943", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "10944", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 35." + }, + { + "id": "10945", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 36." + }, + { + "id": "10946", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 37." + }, + { + "id": "10947", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 38." + }, + { + "id": "10948", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "10949", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 40." + }, + { + "id": "10950", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 41." + }, + { + "id": "10951", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 42." + }, + { + "id": "10952", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 43." + }, + { + "id": "10953", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "10954", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 45." + }, + { + "id": "10955", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 46." + }, + { + "id": "10956", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 47." + }, + { + "id": "10957", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 48." + }, + { + "id": "10958", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "10959", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 50." + }, + { + "id": "10960", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 51." + }, + { + "id": "10961", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 52." + }, + { + "id": "10962", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 53." + }, + { + "id": "10963", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "10964", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 55." + }, + { + "id": "10965", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 56." + }, + { + "id": "10966", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 57." + }, + { + "id": "10967", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 58." + }, + { + "id": "10968", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "10969", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 60." + }, + { + "id": "10970", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 61." + }, + { + "id": "10971", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 62." + }, + { + "id": "10972", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 63." + }, + { + "id": "10973", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "10974", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 65." + }, + { + "id": "10975", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 66." + }, + { + "id": "10976", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 67." + }, + { + "id": "10977", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 68." + }, + { + "id": "10978", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "10979", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 70." + }, + { + "id": "10980", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 71." + }, + { + "id": "10981", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 72." + }, + { + "id": "10982", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 73." + }, + { + "id": "10983", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "10984", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 75." + }, + { + "id": "10985", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 76." + }, + { + "id": "10986", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 77." + }, + { + "id": "10987", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 78." + }, + { + "id": "10988", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "10989", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 80." + }, + { + "id": "10990", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 81." + }, + { + "id": "10991", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 82." + }, + { + "id": "10992", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 83." + }, + { + "id": "10993", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "10994", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 85." + }, + { + "id": "10995", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 86." + }, + { + "id": "10996", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 87." + }, + { + "id": "10997", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 88." + }, + { + "id": "10998", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "10999", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 90." + }, + { + "id": "11000", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 91." + }, + { + "id": "11001", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 92." + }, + { + "id": "11002", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 93." + }, + { + "id": "11003", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "11004", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 95." + }, + { + "id": "11005", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 96." + }, + { + "id": "11006", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 97." + }, + { + "id": "11007", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 98." + }, + { + "id": "11008", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "11009", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 100." + }, + { + "id": "11010", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 101." + }, + { + "id": "11011", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 102." + }, + { + "id": "11012", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 103." + }, + { + "id": "11013", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "11014", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 105." + }, + { + "id": "11015", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 106." + }, + { + "id": "11016", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 107." + }, + { + "id": "11017", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 108." + }, + { + "id": "11018", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "11019", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 110." + }, + { + "id": "11020", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 111." + }, + { + "id": "11021", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 112." + }, + { + "id": "11022", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 113." + }, + { + "id": "11023", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "11024", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 115." + }, + { + "id": "11025", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 116." + }, + { + "id": "11026", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 117." + }, + { + "id": "11027", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 118." + }, + { + "id": "11028", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "11029", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 120." + }, + { + "id": "11030", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 121." + }, + { + "id": "11031", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 122." + }, + { + "id": "11032", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 123." + }, + { + "id": "11033", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "11034", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 125." + }, + { + "id": "11035", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 126." + }, + { + "id": "11036", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 127." + }, + { + "id": "11037", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 128." + }, + { + "id": "11038", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "11039", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 130." + }, + { + "id": "11040", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 131." + }, + { + "id": "11041", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 132." + }, + { + "id": "11042", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 133." + }, + { + "id": "11043", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "11044", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 135." + }, + { + "id": "11045", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 136." + }, + { + "id": "11046", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 137." + }, + { + "id": "11047", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 138." + }, + { + "id": "11048", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "11049", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 140." + }, + { + "id": "11050", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 141." + }, + { + "id": "11051", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 142." + }, + { + "id": "11052", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 143." + }, + { + "id": "11053", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "11054", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 145." + }, + { + "id": "11055", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 146." + }, + { + "id": "11056", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 147." + }, + { + "id": "11057", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 148." + }, + { + "id": "11058", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "11059", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 150." + }, + { + "id": "11060", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 151." + }, + { + "id": "11061", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 152." + }, + { + "id": "11062", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 153." + }, + { + "id": "11063", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "11064", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 155." + }, + { + "id": "11065", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 156." + }, + { + "id": "11066", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 157." + }, + { + "id": "11067", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 158." + }, + { + "id": "11068", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "11069", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 160." + }, + { + "id": "11070", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 161." + }, + { + "id": "11071", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 162." + }, + { + "id": "11072", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 163." + }, + { + "id": "11073", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "11074", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 165." + }, + { + "id": "11075", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 166." + }, + { + "id": "11076", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 167." + }, + { + "id": "11077", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 168." + }, + { + "id": "11078", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "11079", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 170." + }, + { + "id": "11080", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 171." + }, + { + "id": "11081", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 172." + }, + { + "id": "11082", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 173." + }, + { + "id": "11083", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "11084", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 175." + }, + { + "id": "11085", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 176." + }, + { + "id": "11086", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 177." + }, + { + "id": "11087", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 178." + }, + { + "id": "11088", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "11089", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 180." + }, + { + "id": "11090", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 181." + }, + { + "id": "11091", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 182." + }, + { + "id": "11092", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 183." + }, + { + "id": "11093", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "11094", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 185." + }, + { + "id": "11095", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 186." + }, + { + "id": "11096", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 187." + }, + { + "id": "11097", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 188." + }, + { + "id": "11098", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "11099", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 190." + }, + { + "id": "11100", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 191." + }, + { + "id": "11101", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 192." + }, + { + "id": "11102", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 193." + }, + { + "id": "11103", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "11104", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 195." + }, + { + "id": "11105", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 196." + }, + { + "id": "11106", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 197." + }, + { + "id": "11107", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 198." + }, + { + "id": "11108", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "11109", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 200." + }, + { + "id": "11110", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 201." + }, + { + "id": "11111", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 202." + }, + { + "id": "11112", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 203." + }, + { + "id": "11113", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "11114", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 205." + }, + { + "id": "11115", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 206." + }, + { + "id": "11116", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 207." + }, + { + "id": "11117", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 208." + }, + { + "id": "11118", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "11119", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 210." + }, + { + "id": "11120", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 211." + }, + { + "id": "11121", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 212." + }, + { + "id": "11122", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 213." + }, + { + "id": "11123", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "11124", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 215." + }, + { + "id": "11125", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 216." + }, + { + "id": "11126", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 217." + }, + { + "id": "11127", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 218." + }, + { + "id": "11128", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "11129", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 220." + }, + { + "id": "11130", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 221." + }, + { + "id": "11131", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 222." + }, + { + "id": "11132", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 223." + }, + { + "id": "11133", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "11134", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 225." + }, + { + "id": "11135", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 226." + }, + { + "id": "11136", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 227." + }, + { + "id": "11137", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 228." + }, + { + "id": "11138", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "11139", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 230." + }, + { + "id": "11140", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 231." + }, + { + "id": "11141", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 232." + }, + { + "id": "11142", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 233." + }, + { + "id": "11143", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "11144", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 235." + }, + { + "id": "11145", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 236." + }, + { + "id": "11146", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 237." + }, + { + "id": "11147", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 238." + }, + { + "id": "11148", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "11149", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 240." + }, + { + "id": "11150", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 241." + }, + { + "id": "11151", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 242." + }, + { + "id": "11152", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 243." + }, + { + "id": "11153", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "11154", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 245." + }, + { + "id": "11155", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 246." + }, + { + "id": "11156", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 247." + }, + { + "id": "11157", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 248." + }, + { + "id": "11158", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "11159", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 250." + }, + { + "id": "11160", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 251." + }, + { + "id": "11161", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 252." + }, + { + "id": "11162", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 253." + }, + { + "id": "11163", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "11164", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 255." + }, + { + "id": "11165", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 256." + }, + { + "id": "11166", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 257." + }, + { + "id": "11167", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 258." + }, + { + "id": "11168", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "11169", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 260." + }, + { + "id": "11170", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 261." + }, + { + "id": "11171", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 262." + }, + { + "id": "11172", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 263." + }, + { + "id": "11173", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "11174", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 265." + }, + { + "id": "11175", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 266." + }, + { + "id": "11176", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 267." + }, + { + "id": "11177", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 268." + }, + { + "id": "11178", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "11179", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 270." + }, + { + "id": "11180", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 271." + }, + { + "id": "11181", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 272." + }, + { + "id": "11182", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 273." + }, + { + "id": "11183", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "11184", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 275." + }, + { + "id": "11185", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 276." + }, + { + "id": "11186", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 277." + }, + { + "id": "11187", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 278." + }, + { + "id": "11188", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "11189", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 280." + }, + { + "id": "11190", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 281." + }, + { + "id": "11191", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 282." + }, + { + "id": "11192", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 283." + }, + { + "id": "11193", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "11194", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 285." + }, + { + "id": "11195", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 286." + }, + { + "id": "11196", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 287." + }, + { + "id": "11197", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 288." + }, + { + "id": "11198", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "11199", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 290." + }, + { + "id": "11200", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 291." + }, + { + "id": "11201", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 292." + }, + { + "id": "11202", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 293." + }, + { + "id": "11203", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "11204", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund: agenda only; no decision yet. Thread 295." + }, + { + "id": "11205", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen invoice refund? Please add this question to the meeting. Thread 296." + }, + { + "id": "11206", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 297." + }, + { + "id": "11207", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 298." + }, + { + "id": "11208", + "chatId": "104", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "11209", + "chatId": "105", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Harbor release freeze decision: stop deployments until the database check passes." + }, + { + "id": "11210", + "chatId": "105", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "The Harbor release freeze owner is Oren; the decision is recorded above." + }, + { + "id": "11211", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 0." + }, + { + "id": "11212", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 1." + }, + { + "id": "11213", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 2." + }, + { + "id": "11214", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 3." + }, + { + "id": "11215", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "11216", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 5." + }, + { + "id": "11217", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 6." + }, + { + "id": "11218", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 7." + }, + { + "id": "11219", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 8." + }, + { + "id": "11220", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "11221", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 10." + }, + { + "id": "11222", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 11." + }, + { + "id": "11223", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 12." + }, + { + "id": "11224", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 13." + }, + { + "id": "11225", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "11226", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 15." + }, + { + "id": "11227", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 16." + }, + { + "id": "11228", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 17." + }, + { + "id": "11229", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 18." + }, + { + "id": "11230", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "11231", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 20." + }, + { + "id": "11232", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 21." + }, + { + "id": "11233", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 22." + }, + { + "id": "11234", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 23." + }, + { + "id": "11235", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "11236", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 25." + }, + { + "id": "11237", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 26." + }, + { + "id": "11238", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 27." + }, + { + "id": "11239", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 28." + }, + { + "id": "11240", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "11241", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 30." + }, + { + "id": "11242", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 31." + }, + { + "id": "11243", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 32." + }, + { + "id": "11244", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 33." + }, + { + "id": "11245", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "11246", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 35." + }, + { + "id": "11247", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 36." + }, + { + "id": "11248", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 37." + }, + { + "id": "11249", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 38." + }, + { + "id": "11250", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "11251", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 40." + }, + { + "id": "11252", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 41." + }, + { + "id": "11253", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 42." + }, + { + "id": "11254", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 43." + }, + { + "id": "11255", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "11256", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 45." + }, + { + "id": "11257", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 46." + }, + { + "id": "11258", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 47." + }, + { + "id": "11259", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 48." + }, + { + "id": "11260", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "11261", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 50." + }, + { + "id": "11262", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 51." + }, + { + "id": "11263", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 52." + }, + { + "id": "11264", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 53." + }, + { + "id": "11265", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "11266", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 55." + }, + { + "id": "11267", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 56." + }, + { + "id": "11268", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 57." + }, + { + "id": "11269", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 58." + }, + { + "id": "11270", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "11271", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 60." + }, + { + "id": "11272", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 61." + }, + { + "id": "11273", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 62." + }, + { + "id": "11274", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 63." + }, + { + "id": "11275", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "11276", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 65." + }, + { + "id": "11277", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 66." + }, + { + "id": "11278", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 67." + }, + { + "id": "11279", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 68." + }, + { + "id": "11280", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "11281", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 70." + }, + { + "id": "11282", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 71." + }, + { + "id": "11283", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 72." + }, + { + "id": "11284", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 73." + }, + { + "id": "11285", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "11286", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 75." + }, + { + "id": "11287", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 76." + }, + { + "id": "11288", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 77." + }, + { + "id": "11289", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 78." + }, + { + "id": "11290", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "11291", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 80." + }, + { + "id": "11292", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 81." + }, + { + "id": "11293", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 82." + }, + { + "id": "11294", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 83." + }, + { + "id": "11295", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "11296", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 85." + }, + { + "id": "11297", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 86." + }, + { + "id": "11298", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 87." + }, + { + "id": "11299", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 88." + }, + { + "id": "11300", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "11301", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 90." + }, + { + "id": "11302", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 91." + }, + { + "id": "11303", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 92." + }, + { + "id": "11304", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 93." + }, + { + "id": "11305", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "11306", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 95." + }, + { + "id": "11307", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 96." + }, + { + "id": "11308", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 97." + }, + { + "id": "11309", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 98." + }, + { + "id": "11310", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "11311", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 100." + }, + { + "id": "11312", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 101." + }, + { + "id": "11313", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 102." + }, + { + "id": "11314", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 103." + }, + { + "id": "11315", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "11316", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 105." + }, + { + "id": "11317", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 106." + }, + { + "id": "11318", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 107." + }, + { + "id": "11319", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 108." + }, + { + "id": "11320", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "11321", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 110." + }, + { + "id": "11322", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 111." + }, + { + "id": "11323", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 112." + }, + { + "id": "11324", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 113." + }, + { + "id": "11325", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "11326", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 115." + }, + { + "id": "11327", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 116." + }, + { + "id": "11328", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 117." + }, + { + "id": "11329", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 118." + }, + { + "id": "11330", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "11331", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 120." + }, + { + "id": "11332", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 121." + }, + { + "id": "11333", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 122." + }, + { + "id": "11334", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 123." + }, + { + "id": "11335", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "11336", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 125." + }, + { + "id": "11337", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 126." + }, + { + "id": "11338", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 127." + }, + { + "id": "11339", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 128." + }, + { + "id": "11340", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "11341", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 130." + }, + { + "id": "11342", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 131." + }, + { + "id": "11343", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 132." + }, + { + "id": "11344", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 133." + }, + { + "id": "11345", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "11346", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 135." + }, + { + "id": "11347", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 136." + }, + { + "id": "11348", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 137." + }, + { + "id": "11349", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 138." + }, + { + "id": "11350", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "11351", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 140." + }, + { + "id": "11352", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 141." + }, + { + "id": "11353", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 142." + }, + { + "id": "11354", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 143." + }, + { + "id": "11355", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "11356", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 145." + }, + { + "id": "11357", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 146." + }, + { + "id": "11358", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 147." + }, + { + "id": "11359", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 148." + }, + { + "id": "11360", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "11361", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 150." + }, + { + "id": "11362", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 151." + }, + { + "id": "11363", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 152." + }, + { + "id": "11364", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 153." + }, + { + "id": "11365", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "11366", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 155." + }, + { + "id": "11367", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 156." + }, + { + "id": "11368", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 157." + }, + { + "id": "11369", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 158." + }, + { + "id": "11370", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "11371", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 160." + }, + { + "id": "11372", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 161." + }, + { + "id": "11373", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 162." + }, + { + "id": "11374", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 163." + }, + { + "id": "11375", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "11376", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 165." + }, + { + "id": "11377", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 166." + }, + { + "id": "11378", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 167." + }, + { + "id": "11379", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 168." + }, + { + "id": "11380", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "11381", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 170." + }, + { + "id": "11382", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 171." + }, + { + "id": "11383", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 172." + }, + { + "id": "11384", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 173." + }, + { + "id": "11385", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "11386", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 175." + }, + { + "id": "11387", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 176." + }, + { + "id": "11388", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 177." + }, + { + "id": "11389", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 178." + }, + { + "id": "11390", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "11391", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 180." + }, + { + "id": "11392", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 181." + }, + { + "id": "11393", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 182." + }, + { + "id": "11394", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 183." + }, + { + "id": "11395", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "11396", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 185." + }, + { + "id": "11397", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 186." + }, + { + "id": "11398", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 187." + }, + { + "id": "11399", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 188." + }, + { + "id": "11400", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "11401", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 190." + }, + { + "id": "11402", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 191." + }, + { + "id": "11403", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 192." + }, + { + "id": "11404", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 193." + }, + { + "id": "11405", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "11406", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 195." + }, + { + "id": "11407", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 196." + }, + { + "id": "11408", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 197." + }, + { + "id": "11409", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 198." + }, + { + "id": "11410", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "11411", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 200." + }, + { + "id": "11412", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 201." + }, + { + "id": "11413", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 202." + }, + { + "id": "11414", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 203." + }, + { + "id": "11415", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "11416", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 205." + }, + { + "id": "11417", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 206." + }, + { + "id": "11418", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 207." + }, + { + "id": "11419", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 208." + }, + { + "id": "11420", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "11421", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 210." + }, + { + "id": "11422", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 211." + }, + { + "id": "11423", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 212." + }, + { + "id": "11424", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 213." + }, + { + "id": "11425", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "11426", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 215." + }, + { + "id": "11427", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 216." + }, + { + "id": "11428", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 217." + }, + { + "id": "11429", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 218." + }, + { + "id": "11430", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "11431", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 220." + }, + { + "id": "11432", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 221." + }, + { + "id": "11433", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 222." + }, + { + "id": "11434", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 223." + }, + { + "id": "11435", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "11436", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 225." + }, + { + "id": "11437", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 226." + }, + { + "id": "11438", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 227." + }, + { + "id": "11439", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 228." + }, + { + "id": "11440", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "11441", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 230." + }, + { + "id": "11442", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 231." + }, + { + "id": "11443", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 232." + }, + { + "id": "11444", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 233." + }, + { + "id": "11445", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "11446", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 235." + }, + { + "id": "11447", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 236." + }, + { + "id": "11448", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 237." + }, + { + "id": "11449", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 238." + }, + { + "id": "11450", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "11451", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 240." + }, + { + "id": "11452", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 241." + }, + { + "id": "11453", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 242." + }, + { + "id": "11454", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 243." + }, + { + "id": "11455", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "11456", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 245." + }, + { + "id": "11457", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 246." + }, + { + "id": "11458", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 247." + }, + { + "id": "11459", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 248." + }, + { + "id": "11460", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "11461", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 250." + }, + { + "id": "11462", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 251." + }, + { + "id": "11463", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 252." + }, + { + "id": "11464", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 253." + }, + { + "id": "11465", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "11466", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 255." + }, + { + "id": "11467", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 256." + }, + { + "id": "11468", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 257." + }, + { + "id": "11469", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 258." + }, + { + "id": "11470", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "11471", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 260." + }, + { + "id": "11472", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 261." + }, + { + "id": "11473", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 262." + }, + { + "id": "11474", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 263." + }, + { + "id": "11475", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "11476", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 265." + }, + { + "id": "11477", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 266." + }, + { + "id": "11478", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 267." + }, + { + "id": "11479", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 268." + }, + { + "id": "11480", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "11481", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 270." + }, + { + "id": "11482", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 271." + }, + { + "id": "11483", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 272." + }, + { + "id": "11484", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 273." + }, + { + "id": "11485", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "11486", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 275." + }, + { + "id": "11487", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 276." + }, + { + "id": "11488", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 277." + }, + { + "id": "11489", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 278." + }, + { + "id": "11490", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "11491", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 280." + }, + { + "id": "11492", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 281." + }, + { + "id": "11493", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 282." + }, + { + "id": "11494", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 283." + }, + { + "id": "11495", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "11496", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 285." + }, + { + "id": "11497", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 286." + }, + { + "id": "11498", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 287." + }, + { + "id": "11499", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 288." + }, + { + "id": "11500", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "11501", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 290." + }, + { + "id": "11502", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 291." + }, + { + "id": "11503", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 292." + }, + { + "id": "11504", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 293." + }, + { + "id": "11505", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "11506", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze: agenda only; no decision yet. Thread 295." + }, + { + "id": "11507", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor release freeze? Please add this question to the meeting. Thread 296." + }, + { + "id": "11508", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 297." + }, + { + "id": "11509", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 298." + }, + { + "id": "11510", + "chatId": "105", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "11511", + "chatId": "106", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Maple archive restore decision: recover the March snapshot into a separate workspace." + }, + { + "id": "11512", + "chatId": "106", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "The Maple archive restore owner is Ada; the decision is recorded above." + }, + { + "id": "11513", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 0." + }, + { + "id": "11514", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 1." + }, + { + "id": "11515", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 2." + }, + { + "id": "11516", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 3." + }, + { + "id": "11517", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "11518", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 5." + }, + { + "id": "11519", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 6." + }, + { + "id": "11520", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 7." + }, + { + "id": "11521", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 8." + }, + { + "id": "11522", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "11523", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 10." + }, + { + "id": "11524", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 11." + }, + { + "id": "11525", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 12." + }, + { + "id": "11526", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 13." + }, + { + "id": "11527", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "11528", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 15." + }, + { + "id": "11529", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 16." + }, + { + "id": "11530", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 17." + }, + { + "id": "11531", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 18." + }, + { + "id": "11532", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "11533", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 20." + }, + { + "id": "11534", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 21." + }, + { + "id": "11535", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 22." + }, + { + "id": "11536", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 23." + }, + { + "id": "11537", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "11538", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 25." + }, + { + "id": "11539", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 26." + }, + { + "id": "11540", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 27." + }, + { + "id": "11541", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 28." + }, + { + "id": "11542", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "11543", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 30." + }, + { + "id": "11544", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 31." + }, + { + "id": "11545", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 32." + }, + { + "id": "11546", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 33." + }, + { + "id": "11547", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "11548", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 35." + }, + { + "id": "11549", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 36." + }, + { + "id": "11550", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 37." + }, + { + "id": "11551", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 38." + }, + { + "id": "11552", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "11553", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 40." + }, + { + "id": "11554", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 41." + }, + { + "id": "11555", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 42." + }, + { + "id": "11556", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 43." + }, + { + "id": "11557", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "11558", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 45." + }, + { + "id": "11559", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 46." + }, + { + "id": "11560", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 47." + }, + { + "id": "11561", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 48." + }, + { + "id": "11562", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "11563", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 50." + }, + { + "id": "11564", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 51." + }, + { + "id": "11565", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 52." + }, + { + "id": "11566", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 53." + }, + { + "id": "11567", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "11568", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 55." + }, + { + "id": "11569", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 56." + }, + { + "id": "11570", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 57." + }, + { + "id": "11571", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 58." + }, + { + "id": "11572", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "11573", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 60." + }, + { + "id": "11574", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 61." + }, + { + "id": "11575", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 62." + }, + { + "id": "11576", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 63." + }, + { + "id": "11577", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "11578", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 65." + }, + { + "id": "11579", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 66." + }, + { + "id": "11580", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 67." + }, + { + "id": "11581", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 68." + }, + { + "id": "11582", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "11583", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 70." + }, + { + "id": "11584", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 71." + }, + { + "id": "11585", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 72." + }, + { + "id": "11586", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 73." + }, + { + "id": "11587", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "11588", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 75." + }, + { + "id": "11589", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 76." + }, + { + "id": "11590", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 77." + }, + { + "id": "11591", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 78." + }, + { + "id": "11592", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "11593", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 80." + }, + { + "id": "11594", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 81." + }, + { + "id": "11595", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 82." + }, + { + "id": "11596", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 83." + }, + { + "id": "11597", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "11598", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 85." + }, + { + "id": "11599", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 86." + }, + { + "id": "11600", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 87." + }, + { + "id": "11601", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 88." + }, + { + "id": "11602", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "11603", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 90." + }, + { + "id": "11604", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 91." + }, + { + "id": "11605", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 92." + }, + { + "id": "11606", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 93." + }, + { + "id": "11607", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "11608", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 95." + }, + { + "id": "11609", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 96." + }, + { + "id": "11610", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 97." + }, + { + "id": "11611", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 98." + }, + { + "id": "11612", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "11613", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 100." + }, + { + "id": "11614", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 101." + }, + { + "id": "11615", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 102." + }, + { + "id": "11616", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 103." + }, + { + "id": "11617", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "11618", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 105." + }, + { + "id": "11619", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 106." + }, + { + "id": "11620", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 107." + }, + { + "id": "11621", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 108." + }, + { + "id": "11622", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "11623", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 110." + }, + { + "id": "11624", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 111." + }, + { + "id": "11625", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 112." + }, + { + "id": "11626", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 113." + }, + { + "id": "11627", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "11628", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 115." + }, + { + "id": "11629", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 116." + }, + { + "id": "11630", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 117." + }, + { + "id": "11631", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 118." + }, + { + "id": "11632", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "11633", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 120." + }, + { + "id": "11634", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 121." + }, + { + "id": "11635", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 122." + }, + { + "id": "11636", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 123." + }, + { + "id": "11637", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "11638", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 125." + }, + { + "id": "11639", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 126." + }, + { + "id": "11640", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 127." + }, + { + "id": "11641", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 128." + }, + { + "id": "11642", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "11643", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 130." + }, + { + "id": "11644", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 131." + }, + { + "id": "11645", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 132." + }, + { + "id": "11646", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 133." + }, + { + "id": "11647", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "11648", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 135." + }, + { + "id": "11649", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 136." + }, + { + "id": "11650", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 137." + }, + { + "id": "11651", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 138." + }, + { + "id": "11652", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "11653", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 140." + }, + { + "id": "11654", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 141." + }, + { + "id": "11655", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 142." + }, + { + "id": "11656", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 143." + }, + { + "id": "11657", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "11658", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 145." + }, + { + "id": "11659", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 146." + }, + { + "id": "11660", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 147." + }, + { + "id": "11661", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 148." + }, + { + "id": "11662", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "11663", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 150." + }, + { + "id": "11664", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 151." + }, + { + "id": "11665", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 152." + }, + { + "id": "11666", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 153." + }, + { + "id": "11667", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "11668", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 155." + }, + { + "id": "11669", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 156." + }, + { + "id": "11670", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 157." + }, + { + "id": "11671", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 158." + }, + { + "id": "11672", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "11673", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 160." + }, + { + "id": "11674", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 161." + }, + { + "id": "11675", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 162." + }, + { + "id": "11676", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 163." + }, + { + "id": "11677", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "11678", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 165." + }, + { + "id": "11679", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 166." + }, + { + "id": "11680", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 167." + }, + { + "id": "11681", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 168." + }, + { + "id": "11682", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "11683", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 170." + }, + { + "id": "11684", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 171." + }, + { + "id": "11685", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 172." + }, + { + "id": "11686", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 173." + }, + { + "id": "11687", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "11688", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 175." + }, + { + "id": "11689", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 176." + }, + { + "id": "11690", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 177." + }, + { + "id": "11691", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 178." + }, + { + "id": "11692", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "11693", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 180." + }, + { + "id": "11694", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 181." + }, + { + "id": "11695", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 182." + }, + { + "id": "11696", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 183." + }, + { + "id": "11697", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "11698", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 185." + }, + { + "id": "11699", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 186." + }, + { + "id": "11700", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 187." + }, + { + "id": "11701", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 188." + }, + { + "id": "11702", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "11703", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 190." + }, + { + "id": "11704", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 191." + }, + { + "id": "11705", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 192." + }, + { + "id": "11706", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 193." + }, + { + "id": "11707", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "11708", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 195." + }, + { + "id": "11709", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 196." + }, + { + "id": "11710", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 197." + }, + { + "id": "11711", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 198." + }, + { + "id": "11712", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "11713", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 200." + }, + { + "id": "11714", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 201." + }, + { + "id": "11715", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 202." + }, + { + "id": "11716", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 203." + }, + { + "id": "11717", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "11718", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 205." + }, + { + "id": "11719", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 206." + }, + { + "id": "11720", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 207." + }, + { + "id": "11721", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 208." + }, + { + "id": "11722", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "11723", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 210." + }, + { + "id": "11724", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 211." + }, + { + "id": "11725", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 212." + }, + { + "id": "11726", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 213." + }, + { + "id": "11727", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "11728", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 215." + }, + { + "id": "11729", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 216." + }, + { + "id": "11730", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 217." + }, + { + "id": "11731", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 218." + }, + { + "id": "11732", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "11733", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 220." + }, + { + "id": "11734", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 221." + }, + { + "id": "11735", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 222." + }, + { + "id": "11736", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 223." + }, + { + "id": "11737", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "11738", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 225." + }, + { + "id": "11739", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 226." + }, + { + "id": "11740", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 227." + }, + { + "id": "11741", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 228." + }, + { + "id": "11742", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "11743", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 230." + }, + { + "id": "11744", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 231." + }, + { + "id": "11745", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 232." + }, + { + "id": "11746", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 233." + }, + { + "id": "11747", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "11748", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 235." + }, + { + "id": "11749", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 236." + }, + { + "id": "11750", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 237." + }, + { + "id": "11751", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 238." + }, + { + "id": "11752", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "11753", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 240." + }, + { + "id": "11754", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 241." + }, + { + "id": "11755", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 242." + }, + { + "id": "11756", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 243." + }, + { + "id": "11757", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "11758", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 245." + }, + { + "id": "11759", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 246." + }, + { + "id": "11760", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 247." + }, + { + "id": "11761", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 248." + }, + { + "id": "11762", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "11763", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 250." + }, + { + "id": "11764", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 251." + }, + { + "id": "11765", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 252." + }, + { + "id": "11766", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 253." + }, + { + "id": "11767", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "11768", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 255." + }, + { + "id": "11769", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 256." + }, + { + "id": "11770", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 257." + }, + { + "id": "11771", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 258." + }, + { + "id": "11772", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "11773", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 260." + }, + { + "id": "11774", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 261." + }, + { + "id": "11775", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 262." + }, + { + "id": "11776", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 263." + }, + { + "id": "11777", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "11778", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 265." + }, + { + "id": "11779", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 266." + }, + { + "id": "11780", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 267." + }, + { + "id": "11781", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 268." + }, + { + "id": "11782", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "11783", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 270." + }, + { + "id": "11784", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 271." + }, + { + "id": "11785", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 272." + }, + { + "id": "11786", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 273." + }, + { + "id": "11787", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "11788", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 275." + }, + { + "id": "11789", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 276." + }, + { + "id": "11790", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 277." + }, + { + "id": "11791", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 278." + }, + { + "id": "11792", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "11793", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 280." + }, + { + "id": "11794", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 281." + }, + { + "id": "11795", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 282." + }, + { + "id": "11796", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 283." + }, + { + "id": "11797", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "11798", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 285." + }, + { + "id": "11799", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 286." + }, + { + "id": "11800", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 287." + }, + { + "id": "11801", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 288." + }, + { + "id": "11802", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "11803", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 290." + }, + { + "id": "11804", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 291." + }, + { + "id": "11805", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 292." + }, + { + "id": "11806", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 293." + }, + { + "id": "11807", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "11808", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore: agenda only; no decision yet. Thread 295." + }, + { + "id": "11809", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple archive restore? Please add this question to the meeting. Thread 296." + }, + { + "id": "11810", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 297." + }, + { + "id": "11811", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 298." + }, + { + "id": "11812", + "chatId": "106", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "11813", + "chatId": "107", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Вектор миграция отмена: отключить задание и вернуть схему версии 8." + }, + { + "id": "11814", + "chatId": "107", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Вектор миграция отмена проверена, владелец Лев." + }, + { + "id": "11815", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 0." + }, + { + "id": "11816", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 1." + }, + { + "id": "11817", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 2." + }, + { + "id": "11818", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 3." + }, + { + "id": "11819", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "11820", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 5." + }, + { + "id": "11821", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 6." + }, + { + "id": "11822", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 7." + }, + { + "id": "11823", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 8." + }, + { + "id": "11824", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "11825", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 10." + }, + { + "id": "11826", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 11." + }, + { + "id": "11827", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 12." + }, + { + "id": "11828", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 13." + }, + { + "id": "11829", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "11830", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 15." + }, + { + "id": "11831", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 16." + }, + { + "id": "11832", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 17." + }, + { + "id": "11833", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 18." + }, + { + "id": "11834", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "11835", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 20." + }, + { + "id": "11836", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 21." + }, + { + "id": "11837", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 22." + }, + { + "id": "11838", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 23." + }, + { + "id": "11839", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "11840", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 25." + }, + { + "id": "11841", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 26." + }, + { + "id": "11842", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 27." + }, + { + "id": "11843", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 28." + }, + { + "id": "11844", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "11845", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 30." + }, + { + "id": "11846", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 31." + }, + { + "id": "11847", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 32." + }, + { + "id": "11848", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 33." + }, + { + "id": "11849", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "11850", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 35." + }, + { + "id": "11851", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 36." + }, + { + "id": "11852", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 37." + }, + { + "id": "11853", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 38." + }, + { + "id": "11854", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "11855", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 40." + }, + { + "id": "11856", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 41." + }, + { + "id": "11857", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 42." + }, + { + "id": "11858", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 43." + }, + { + "id": "11859", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "11860", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 45." + }, + { + "id": "11861", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 46." + }, + { + "id": "11862", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 47." + }, + { + "id": "11863", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 48." + }, + { + "id": "11864", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "11865", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 50." + }, + { + "id": "11866", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 51." + }, + { + "id": "11867", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 52." + }, + { + "id": "11868", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 53." + }, + { + "id": "11869", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "11870", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 55." + }, + { + "id": "11871", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 56." + }, + { + "id": "11872", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 57." + }, + { + "id": "11873", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 58." + }, + { + "id": "11874", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "11875", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 60." + }, + { + "id": "11876", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 61." + }, + { + "id": "11877", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 62." + }, + { + "id": "11878", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 63." + }, + { + "id": "11879", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "11880", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 65." + }, + { + "id": "11881", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 66." + }, + { + "id": "11882", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 67." + }, + { + "id": "11883", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 68." + }, + { + "id": "11884", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "11885", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 70." + }, + { + "id": "11886", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 71." + }, + { + "id": "11887", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 72." + }, + { + "id": "11888", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 73." + }, + { + "id": "11889", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "11890", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 75." + }, + { + "id": "11891", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 76." + }, + { + "id": "11892", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 77." + }, + { + "id": "11893", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 78." + }, + { + "id": "11894", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "11895", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 80." + }, + { + "id": "11896", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 81." + }, + { + "id": "11897", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 82." + }, + { + "id": "11898", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 83." + }, + { + "id": "11899", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "11900", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 85." + }, + { + "id": "11901", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 86." + }, + { + "id": "11902", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 87." + }, + { + "id": "11903", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 88." + }, + { + "id": "11904", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "11905", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 90." + }, + { + "id": "11906", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 91." + }, + { + "id": "11907", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 92." + }, + { + "id": "11908", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 93." + }, + { + "id": "11909", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "11910", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 95." + }, + { + "id": "11911", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 96." + }, + { + "id": "11912", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 97." + }, + { + "id": "11913", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 98." + }, + { + "id": "11914", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "11915", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 100." + }, + { + "id": "11916", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 101." + }, + { + "id": "11917", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 102." + }, + { + "id": "11918", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 103." + }, + { + "id": "11919", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "11920", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 105." + }, + { + "id": "11921", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 106." + }, + { + "id": "11922", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 107." + }, + { + "id": "11923", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 108." + }, + { + "id": "11924", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "11925", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 110." + }, + { + "id": "11926", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 111." + }, + { + "id": "11927", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 112." + }, + { + "id": "11928", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 113." + }, + { + "id": "11929", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "11930", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 115." + }, + { + "id": "11931", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 116." + }, + { + "id": "11932", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 117." + }, + { + "id": "11933", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 118." + }, + { + "id": "11934", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "11935", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 120." + }, + { + "id": "11936", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 121." + }, + { + "id": "11937", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 122." + }, + { + "id": "11938", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 123." + }, + { + "id": "11939", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "11940", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 125." + }, + { + "id": "11941", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 126." + }, + { + "id": "11942", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 127." + }, + { + "id": "11943", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 128." + }, + { + "id": "11944", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "11945", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 130." + }, + { + "id": "11946", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 131." + }, + { + "id": "11947", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 132." + }, + { + "id": "11948", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 133." + }, + { + "id": "11949", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "11950", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 135." + }, + { + "id": "11951", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 136." + }, + { + "id": "11952", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 137." + }, + { + "id": "11953", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 138." + }, + { + "id": "11954", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "11955", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 140." + }, + { + "id": "11956", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 141." + }, + { + "id": "11957", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 142." + }, + { + "id": "11958", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 143." + }, + { + "id": "11959", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "11960", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 145." + }, + { + "id": "11961", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 146." + }, + { + "id": "11962", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 147." + }, + { + "id": "11963", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 148." + }, + { + "id": "11964", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "11965", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 150." + }, + { + "id": "11966", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 151." + }, + { + "id": "11967", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 152." + }, + { + "id": "11968", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 153." + }, + { + "id": "11969", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "11970", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 155." + }, + { + "id": "11971", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 156." + }, + { + "id": "11972", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 157." + }, + { + "id": "11973", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 158." + }, + { + "id": "11974", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "11975", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 160." + }, + { + "id": "11976", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 161." + }, + { + "id": "11977", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 162." + }, + { + "id": "11978", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 163." + }, + { + "id": "11979", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "11980", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 165." + }, + { + "id": "11981", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 166." + }, + { + "id": "11982", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 167." + }, + { + "id": "11983", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 168." + }, + { + "id": "11984", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "11985", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 170." + }, + { + "id": "11986", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 171." + }, + { + "id": "11987", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 172." + }, + { + "id": "11988", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 173." + }, + { + "id": "11989", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "11990", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 175." + }, + { + "id": "11991", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 176." + }, + { + "id": "11992", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 177." + }, + { + "id": "11993", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 178." + }, + { + "id": "11994", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "11995", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 180." + }, + { + "id": "11996", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 181." + }, + { + "id": "11997", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 182." + }, + { + "id": "11998", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 183." + }, + { + "id": "11999", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "12000", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 185." + }, + { + "id": "12001", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 186." + }, + { + "id": "12002", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 187." + }, + { + "id": "12003", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 188." + }, + { + "id": "12004", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "12005", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 190." + }, + { + "id": "12006", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 191." + }, + { + "id": "12007", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 192." + }, + { + "id": "12008", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 193." + }, + { + "id": "12009", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "12010", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 195." + }, + { + "id": "12011", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 196." + }, + { + "id": "12012", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 197." + }, + { + "id": "12013", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 198." + }, + { + "id": "12014", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "12015", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 200." + }, + { + "id": "12016", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 201." + }, + { + "id": "12017", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 202." + }, + { + "id": "12018", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 203." + }, + { + "id": "12019", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "12020", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 205." + }, + { + "id": "12021", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 206." + }, + { + "id": "12022", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 207." + }, + { + "id": "12023", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 208." + }, + { + "id": "12024", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "12025", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 210." + }, + { + "id": "12026", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 211." + }, + { + "id": "12027", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 212." + }, + { + "id": "12028", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 213." + }, + { + "id": "12029", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "12030", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 215." + }, + { + "id": "12031", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 216." + }, + { + "id": "12032", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 217." + }, + { + "id": "12033", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 218." + }, + { + "id": "12034", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "12035", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 220." + }, + { + "id": "12036", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 221." + }, + { + "id": "12037", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 222." + }, + { + "id": "12038", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 223." + }, + { + "id": "12039", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "12040", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 225." + }, + { + "id": "12041", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 226." + }, + { + "id": "12042", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 227." + }, + { + "id": "12043", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 228." + }, + { + "id": "12044", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "12045", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 230." + }, + { + "id": "12046", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 231." + }, + { + "id": "12047", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 232." + }, + { + "id": "12048", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 233." + }, + { + "id": "12049", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "12050", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 235." + }, + { + "id": "12051", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 236." + }, + { + "id": "12052", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 237." + }, + { + "id": "12053", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 238." + }, + { + "id": "12054", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "12055", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 240." + }, + { + "id": "12056", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 241." + }, + { + "id": "12057", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 242." + }, + { + "id": "12058", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 243." + }, + { + "id": "12059", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "12060", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 245." + }, + { + "id": "12061", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 246." + }, + { + "id": "12062", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 247." + }, + { + "id": "12063", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 248." + }, + { + "id": "12064", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "12065", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 250." + }, + { + "id": "12066", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 251." + }, + { + "id": "12067", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 252." + }, + { + "id": "12068", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 253." + }, + { + "id": "12069", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "12070", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 255." + }, + { + "id": "12071", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 256." + }, + { + "id": "12072", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 257." + }, + { + "id": "12073", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 258." + }, + { + "id": "12074", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "12075", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 260." + }, + { + "id": "12076", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 261." + }, + { + "id": "12077", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 262." + }, + { + "id": "12078", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 263." + }, + { + "id": "12079", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "12080", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 265." + }, + { + "id": "12081", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 266." + }, + { + "id": "12082", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 267." + }, + { + "id": "12083", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 268." + }, + { + "id": "12084", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "12085", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 270." + }, + { + "id": "12086", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 271." + }, + { + "id": "12087", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 272." + }, + { + "id": "12088", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 273." + }, + { + "id": "12089", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "12090", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 275." + }, + { + "id": "12091", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 276." + }, + { + "id": "12092", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 277." + }, + { + "id": "12093", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 278." + }, + { + "id": "12094", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "12095", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 280." + }, + { + "id": "12096", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 281." + }, + { + "id": "12097", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 282." + }, + { + "id": "12098", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 283." + }, + { + "id": "12099", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "12100", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 285." + }, + { + "id": "12101", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 286." + }, + { + "id": "12102", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 287." + }, + { + "id": "12103", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 288." + }, + { + "id": "12104", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "12105", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 290." + }, + { + "id": "12106", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 291." + }, + { + "id": "12107", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 292." + }, + { + "id": "12108", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 293." + }, + { + "id": "12109", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "12110", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена: agenda only; no decision yet. Thread 295." + }, + { + "id": "12111", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор миграция отмена? Please add this question to the meeting. Thread 296." + }, + { + "id": "12112", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 297." + }, + { + "id": "12113", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 298." + }, + { + "id": "12114", + "chatId": "107", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "12115", + "chatId": "108", + "senderId": "700", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "Prism renewal discount approved: apply 15 percent to the annual contract." + }, + { + "id": "12116", + "chatId": "108", + "senderId": "701", + "timestamp": "2026-10-02T12:00:00.000Z", + "text": "The Prism renewal discount owner is Tess; approval is recorded above." + }, + { + "id": "12117", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 0." + }, + { + "id": "12118", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 1." + }, + { + "id": "12119", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 2." + }, + { + "id": "12120", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 3." + }, + { + "id": "12121", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 4." + }, + { + "id": "12122", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 5." + }, + { + "id": "12123", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 6." + }, + { + "id": "12124", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 7." + }, + { + "id": "12125", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 8." + }, + { + "id": "12126", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 9." + }, + { + "id": "12127", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 10." + }, + { + "id": "12128", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 11." + }, + { + "id": "12129", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 12." + }, + { + "id": "12130", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 13." + }, + { + "id": "12131", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 14." + }, + { + "id": "12132", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 15." + }, + { + "id": "12133", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 16." + }, + { + "id": "12134", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 17." + }, + { + "id": "12135", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 18." + }, + { + "id": "12136", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 19." + }, + { + "id": "12137", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 20." + }, + { + "id": "12138", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 21." + }, + { + "id": "12139", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 22." + }, + { + "id": "12140", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 23." + }, + { + "id": "12141", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 24." + }, + { + "id": "12142", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 25." + }, + { + "id": "12143", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 26." + }, + { + "id": "12144", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 27." + }, + { + "id": "12145", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 28." + }, + { + "id": "12146", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 29." + }, + { + "id": "12147", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 30." + }, + { + "id": "12148", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 31." + }, + { + "id": "12149", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 32." + }, + { + "id": "12150", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 33." + }, + { + "id": "12151", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 34." + }, + { + "id": "12152", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 35." + }, + { + "id": "12153", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 36." + }, + { + "id": "12154", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 37." + }, + { + "id": "12155", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 38." + }, + { + "id": "12156", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 39." + }, + { + "id": "12157", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 40." + }, + { + "id": "12158", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 41." + }, + { + "id": "12159", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 42." + }, + { + "id": "12160", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 43." + }, + { + "id": "12161", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 44." + }, + { + "id": "12162", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 45." + }, + { + "id": "12163", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 46." + }, + { + "id": "12164", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 47." + }, + { + "id": "12165", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 48." + }, + { + "id": "12166", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 49." + }, + { + "id": "12167", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 50." + }, + { + "id": "12168", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 51." + }, + { + "id": "12169", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 52." + }, + { + "id": "12170", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 53." + }, + { + "id": "12171", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 54." + }, + { + "id": "12172", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 55." + }, + { + "id": "12173", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 56." + }, + { + "id": "12174", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 57." + }, + { + "id": "12175", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 58." + }, + { + "id": "12176", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 59." + }, + { + "id": "12177", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 60." + }, + { + "id": "12178", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 61." + }, + { + "id": "12179", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 62." + }, + { + "id": "12180", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 63." + }, + { + "id": "12181", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 64." + }, + { + "id": "12182", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 65." + }, + { + "id": "12183", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 66." + }, + { + "id": "12184", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 67." + }, + { + "id": "12185", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 68." + }, + { + "id": "12186", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 69." + }, + { + "id": "12187", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 70." + }, + { + "id": "12188", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 71." + }, + { + "id": "12189", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 72." + }, + { + "id": "12190", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 73." + }, + { + "id": "12191", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 74." + }, + { + "id": "12192", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 75." + }, + { + "id": "12193", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 76." + }, + { + "id": "12194", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 77." + }, + { + "id": "12195", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 78." + }, + { + "id": "12196", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 79." + }, + { + "id": "12197", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 80." + }, + { + "id": "12198", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 81." + }, + { + "id": "12199", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 82." + }, + { + "id": "12200", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 83." + }, + { + "id": "12201", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 84." + }, + { + "id": "12202", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 85." + }, + { + "id": "12203", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 86." + }, + { + "id": "12204", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 87." + }, + { + "id": "12205", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 88." + }, + { + "id": "12206", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 89." + }, + { + "id": "12207", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 90." + }, + { + "id": "12208", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 91." + }, + { + "id": "12209", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 92." + }, + { + "id": "12210", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 93." + }, + { + "id": "12211", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 94." + }, + { + "id": "12212", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 95." + }, + { + "id": "12213", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 96." + }, + { + "id": "12214", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 97." + }, + { + "id": "12215", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 98." + }, + { + "id": "12216", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 99." + }, + { + "id": "12217", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 100." + }, + { + "id": "12218", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 101." + }, + { + "id": "12219", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 102." + }, + { + "id": "12220", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 103." + }, + { + "id": "12221", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 104." + }, + { + "id": "12222", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 105." + }, + { + "id": "12223", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 106." + }, + { + "id": "12224", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 107." + }, + { + "id": "12225", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 108." + }, + { + "id": "12226", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 109." + }, + { + "id": "12227", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 110." + }, + { + "id": "12228", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 111." + }, + { + "id": "12229", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 112." + }, + { + "id": "12230", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 113." + }, + { + "id": "12231", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 114." + }, + { + "id": "12232", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 115." + }, + { + "id": "12233", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 116." + }, + { + "id": "12234", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 117." + }, + { + "id": "12235", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 118." + }, + { + "id": "12236", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 119." + }, + { + "id": "12237", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 120." + }, + { + "id": "12238", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 121." + }, + { + "id": "12239", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 122." + }, + { + "id": "12240", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 123." + }, + { + "id": "12241", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 124." + }, + { + "id": "12242", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 125." + }, + { + "id": "12243", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 126." + }, + { + "id": "12244", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 127." + }, + { + "id": "12245", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 128." + }, + { + "id": "12246", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 129." + }, + { + "id": "12247", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 130." + }, + { + "id": "12248", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 131." + }, + { + "id": "12249", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 132." + }, + { + "id": "12250", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 133." + }, + { + "id": "12251", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 134." + }, + { + "id": "12252", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 135." + }, + { + "id": "12253", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 136." + }, + { + "id": "12254", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 137." + }, + { + "id": "12255", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 138." + }, + { + "id": "12256", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 139." + }, + { + "id": "12257", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 140." + }, + { + "id": "12258", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 141." + }, + { + "id": "12259", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 142." + }, + { + "id": "12260", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 143." + }, + { + "id": "12261", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 144." + }, + { + "id": "12262", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 145." + }, + { + "id": "12263", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 146." + }, + { + "id": "12264", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 147." + }, + { + "id": "12265", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 148." + }, + { + "id": "12266", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 149." + }, + { + "id": "12267", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 150." + }, + { + "id": "12268", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 151." + }, + { + "id": "12269", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 152." + }, + { + "id": "12270", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 153." + }, + { + "id": "12271", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 154." + }, + { + "id": "12272", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 155." + }, + { + "id": "12273", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 156." + }, + { + "id": "12274", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 157." + }, + { + "id": "12275", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 158." + }, + { + "id": "12276", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 159." + }, + { + "id": "12277", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 160." + }, + { + "id": "12278", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 161." + }, + { + "id": "12279", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 162." + }, + { + "id": "12280", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 163." + }, + { + "id": "12281", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 164." + }, + { + "id": "12282", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 165." + }, + { + "id": "12283", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 166." + }, + { + "id": "12284", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 167." + }, + { + "id": "12285", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 168." + }, + { + "id": "12286", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 169." + }, + { + "id": "12287", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 170." + }, + { + "id": "12288", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 171." + }, + { + "id": "12289", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 172." + }, + { + "id": "12290", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 173." + }, + { + "id": "12291", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 174." + }, + { + "id": "12292", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 175." + }, + { + "id": "12293", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 176." + }, + { + "id": "12294", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 177." + }, + { + "id": "12295", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 178." + }, + { + "id": "12296", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 179." + }, + { + "id": "12297", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 180." + }, + { + "id": "12298", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 181." + }, + { + "id": "12299", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 182." + }, + { + "id": "12300", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 183." + }, + { + "id": "12301", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 184." + }, + { + "id": "12302", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 185." + }, + { + "id": "12303", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 186." + }, + { + "id": "12304", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 187." + }, + { + "id": "12305", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 188." + }, + { + "id": "12306", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 189." + }, + { + "id": "12307", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 190." + }, + { + "id": "12308", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 191." + }, + { + "id": "12309", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 192." + }, + { + "id": "12310", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 193." + }, + { + "id": "12311", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 194." + }, + { + "id": "12312", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 195." + }, + { + "id": "12313", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 196." + }, + { + "id": "12314", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 197." + }, + { + "id": "12315", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 198." + }, + { + "id": "12316", + "chatId": "999", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 199." + }, + { + "id": "12317", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 200." + }, + { + "id": "12318", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 201." + }, + { + "id": "12319", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 202." + }, + { + "id": "12320", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 203." + }, + { + "id": "12321", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 204." + }, + { + "id": "12322", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 205." + }, + { + "id": "12323", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 206." + }, + { + "id": "12324", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 207." + }, + { + "id": "12325", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 208." + }, + { + "id": "12326", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 209." + }, + { + "id": "12327", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 210." + }, + { + "id": "12328", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 211." + }, + { + "id": "12329", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 212." + }, + { + "id": "12330", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 213." + }, + { + "id": "12331", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 214." + }, + { + "id": "12332", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 215." + }, + { + "id": "12333", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 216." + }, + { + "id": "12334", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 217." + }, + { + "id": "12335", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 218." + }, + { + "id": "12336", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 219." + }, + { + "id": "12337", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 220." + }, + { + "id": "12338", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 221." + }, + { + "id": "12339", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 222." + }, + { + "id": "12340", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 223." + }, + { + "id": "12341", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 224." + }, + { + "id": "12342", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 225." + }, + { + "id": "12343", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 226." + }, + { + "id": "12344", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 227." + }, + { + "id": "12345", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 228." + }, + { + "id": "12346", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 229." + }, + { + "id": "12347", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 230." + }, + { + "id": "12348", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 231." + }, + { + "id": "12349", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 232." + }, + { + "id": "12350", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 233." + }, + { + "id": "12351", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 234." + }, + { + "id": "12352", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 235." + }, + { + "id": "12353", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 236." + }, + { + "id": "12354", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 237." + }, + { + "id": "12355", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 238." + }, + { + "id": "12356", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 239." + }, + { + "id": "12357", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 240." + }, + { + "id": "12358", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 241." + }, + { + "id": "12359", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 242." + }, + { + "id": "12360", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 243." + }, + { + "id": "12361", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 244." + }, + { + "id": "12362", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 245." + }, + { + "id": "12363", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 246." + }, + { + "id": "12364", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 247." + }, + { + "id": "12365", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 248." + }, + { + "id": "12366", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 249." + }, + { + "id": "12367", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 250." + }, + { + "id": "12368", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 251." + }, + { + "id": "12369", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 252." + }, + { + "id": "12370", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 253." + }, + { + "id": "12371", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 254." + }, + { + "id": "12372", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 255." + }, + { + "id": "12373", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 256." + }, + { + "id": "12374", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 257." + }, + { + "id": "12375", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 258." + }, + { + "id": "12376", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 259." + }, + { + "id": "12377", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 260." + }, + { + "id": "12378", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 261." + }, + { + "id": "12379", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 262." + }, + { + "id": "12380", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 263." + }, + { + "id": "12381", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 264." + }, + { + "id": "12382", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 265." + }, + { + "id": "12383", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 266." + }, + { + "id": "12384", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 267." + }, + { + "id": "12385", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 268." + }, + { + "id": "12386", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 269." + }, + { + "id": "12387", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 270." + }, + { + "id": "12388", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 271." + }, + { + "id": "12389", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 272." + }, + { + "id": "12390", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 273." + }, + { + "id": "12391", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 274." + }, + { + "id": "12392", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 275." + }, + { + "id": "12393", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 276." + }, + { + "id": "12394", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 277." + }, + { + "id": "12395", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 278." + }, + { + "id": "12396", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 279." + }, + { + "id": "12397", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 280." + }, + { + "id": "12398", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 281." + }, + { + "id": "12399", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 282." + }, + { + "id": "12400", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 283." + }, + { + "id": "12401", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 284." + }, + { + "id": "12402", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 285." + }, + { + "id": "12403", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 286." + }, + { + "id": "12404", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 287." + }, + { + "id": "12405", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 288." + }, + { + "id": "12406", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 289." + }, + { + "id": "12407", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 290." + }, + { + "id": "12408", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 291." + }, + { + "id": "12409", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 292." + }, + { + "id": "12410", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 293." + }, + { + "id": "12411", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 294." + }, + { + "id": "12412", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount: agenda only; no decision yet. Thread 295." + }, + { + "id": "12413", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism renewal discount? Please add this question to the meeting. Thread 296." + }, + { + "id": "12414", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 297." + }, + { + "id": "12415", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 298." + }, + { + "id": "12416", + "chatId": "108", + "senderId": "702", + "timestamp": "2026-10-03T12:00:00.000Z", + "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 299." + }, + { + "id": "12417", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 0." + }, + { + "id": "12418", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 1." + }, + { + "id": "12419", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 2." + }, + { + "id": "12420", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 3." + }, + { + "id": "12421", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 4." + }, + { + "id": "12422", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 5." + }, + { + "id": "12423", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 6." + }, + { + "id": "12424", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 7." + }, + { + "id": "12425", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 8." + }, + { + "id": "12426", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 9." + }, + { + "id": "12427", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 10." + }, + { + "id": "12428", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 11." + }, + { + "id": "12429", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 12." + }, + { + "id": "12430", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 13." + }, + { + "id": "12431", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 14." + }, + { + "id": "12432", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 15." + }, + { + "id": "12433", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 16." + }, + { + "id": "12434", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 17." + }, + { + "id": "12435", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 18." + }, + { + "id": "12436", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 19." + }, + { + "id": "12437", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 20." + }, + { + "id": "12438", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 21." + }, + { + "id": "12439", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 22." + }, + { + "id": "12440", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 23." + }, + { + "id": "12441", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 24." + }, + { + "id": "12442", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 25." + }, + { + "id": "12443", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 26." + }, + { + "id": "12444", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 27." + }, + { + "id": "12445", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 28." + }, + { + "id": "12446", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 29." + }, + { + "id": "12447", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 30." + }, + { + "id": "12448", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 31." + }, + { + "id": "12449", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 32." + }, + { + "id": "12450", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 33." + }, + { + "id": "12451", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 34." + }, + { + "id": "12452", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 35." + }, + { + "id": "12453", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 36." + }, + { + "id": "12454", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 37." + }, + { + "id": "12455", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 38." + }, + { + "id": "12456", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 39." + }, + { + "id": "12457", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 40." + }, + { + "id": "12458", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 41." + }, + { + "id": "12459", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 42." + }, + { + "id": "12460", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 43." + }, + { + "id": "12461", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 44." + }, + { + "id": "12462", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 45." + }, + { + "id": "12463", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 46." + }, + { + "id": "12464", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 47." + }, + { + "id": "12465", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 48." + }, + { + "id": "12466", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 49." + }, + { + "id": "12467", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 50." + }, + { + "id": "12468", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 51." + }, + { + "id": "12469", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 52." + }, + { + "id": "12470", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 53." + }, + { + "id": "12471", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 54." + }, + { + "id": "12472", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 55." + }, + { + "id": "12473", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 56." + }, + { + "id": "12474", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 57." + }, + { + "id": "12475", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 58." + }, + { + "id": "12476", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 59." + }, + { + "id": "12477", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 60." + }, + { + "id": "12478", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 61." + }, + { + "id": "12479", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 62." + }, + { + "id": "12480", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 63." + }, + { + "id": "12481", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 64." + }, + { + "id": "12482", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 65." + }, + { + "id": "12483", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 66." + }, + { + "id": "12484", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 67." + }, + { + "id": "12485", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 68." + }, + { + "id": "12486", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 69." + }, + { + "id": "12487", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 70." + }, + { + "id": "12488", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 71." + }, + { + "id": "12489", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 72." + }, + { + "id": "12490", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 73." + }, + { + "id": "12491", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 74." + }, + { + "id": "12492", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 75." + }, + { + "id": "12493", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 76." + }, + { + "id": "12494", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 77." + }, + { + "id": "12495", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 78." + }, + { + "id": "12496", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 79." + }, + { + "id": "12497", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 80." + }, + { + "id": "12498", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 81." + }, + { + "id": "12499", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 82." + }, + { + "id": "12500", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 83." + }, + { + "id": "12501", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 84." + }, + { + "id": "12502", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 85." + }, + { + "id": "12503", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 86." + }, + { + "id": "12504", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 87." + }, + { + "id": "12505", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 88." + }, + { + "id": "12506", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 89." + }, + { + "id": "12507", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 90." + }, + { + "id": "12508", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 91." + }, + { + "id": "12509", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 92." + }, + { + "id": "12510", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 93." + }, + { + "id": "12511", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 94." + }, + { + "id": "12512", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 95." + }, + { + "id": "12513", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 96." + }, + { + "id": "12514", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 97." + }, + { + "id": "12515", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 98." + }, + { + "id": "12516", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 99." + }, + { + "id": "12517", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 100." + }, + { + "id": "12518", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 101." + }, + { + "id": "12519", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 102." + }, + { + "id": "12520", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 103." + }, + { + "id": "12521", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 104." + }, + { + "id": "12522", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 105." + }, + { + "id": "12523", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 106." + }, + { + "id": "12524", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 107." + }, + { + "id": "12525", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 108." + }, + { + "id": "12526", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 109." + }, + { + "id": "12527", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 110." + }, + { + "id": "12528", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 111." + }, + { + "id": "12529", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 112." + }, + { + "id": "12530", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 113." + }, + { + "id": "12531", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 114." + }, + { + "id": "12532", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 115." + }, + { + "id": "12533", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 116." + }, + { + "id": "12534", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 117." + }, + { + "id": "12535", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 118." + }, + { + "id": "12536", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 119." + }, + { + "id": "12537", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 120." + }, + { + "id": "12538", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 121." + }, + { + "id": "12539", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 122." + }, + { + "id": "12540", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 123." + }, + { + "id": "12541", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 124." + }, + { + "id": "12542", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 125." + }, + { + "id": "12543", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 126." + }, + { + "id": "12544", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 127." + }, + { + "id": "12545", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 128." + }, + { + "id": "12546", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 129." + }, + { + "id": "12547", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 130." + }, + { + "id": "12548", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 131." + }, + { + "id": "12549", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 132." + }, + { + "id": "12550", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 133." + }, + { + "id": "12551", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 134." + }, + { + "id": "12552", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 135." + }, + { + "id": "12553", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 136." + }, + { + "id": "12554", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 137." + }, + { + "id": "12555", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 138." + }, + { + "id": "12556", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 139." + }, + { + "id": "12557", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 140." + }, + { + "id": "12558", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 141." + }, + { + "id": "12559", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 142." + }, + { + "id": "12560", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 143." + }, + { + "id": "12561", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 144." + }, + { + "id": "12562", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 145." + }, + { + "id": "12563", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 146." + }, + { + "id": "12564", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 147." + }, + { + "id": "12565", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 148." + }, + { + "id": "12566", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 149." + }, + { + "id": "12567", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 150." + }, + { + "id": "12568", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 151." + }, + { + "id": "12569", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 152." + }, + { + "id": "12570", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 153." + }, + { + "id": "12571", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 154." + }, + { + "id": "12572", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 155." + }, + { + "id": "12573", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 156." + }, + { + "id": "12574", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 157." + }, + { + "id": "12575", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 158." + }, + { + "id": "12576", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 159." + }, + { + "id": "12577", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 160." + }, + { + "id": "12578", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 161." + }, + { + "id": "12579", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 162." + }, + { + "id": "12580", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 163." + }, + { + "id": "12581", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 164." + }, + { + "id": "12582", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 165." + }, + { + "id": "12583", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 166." + }, + { + "id": "12584", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 167." + }, + { + "id": "12585", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 168." + }, + { + "id": "12586", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 169." + }, + { + "id": "12587", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 170." + }, + { + "id": "12588", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 171." + }, + { + "id": "12589", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 172." + }, + { + "id": "12590", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 173." + }, + { + "id": "12591", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 174." + }, + { + "id": "12592", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 175." + }, + { + "id": "12593", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 176." + }, + { + "id": "12594", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 177." + }, + { + "id": "12595", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 178." + }, + { + "id": "12596", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 179." + }, + { + "id": "12597", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 180." + }, + { + "id": "12598", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 181." + }, + { + "id": "12599", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 182." + }, + { + "id": "12600", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 183." + }, + { + "id": "12601", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 184." + }, + { + "id": "12602", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 185." + }, + { + "id": "12603", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 186." + }, + { + "id": "12604", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 187." + }, + { + "id": "12605", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 188." + }, + { + "id": "12606", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 189." + }, + { + "id": "12607", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 190." + }, + { + "id": "12608", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 191." + }, + { + "id": "12609", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 192." + }, + { + "id": "12610", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 193." + }, + { + "id": "12611", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 194." + }, + { + "id": "12612", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 195." + }, + { + "id": "12613", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 196." + }, + { + "id": "12614", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 197." + }, + { + "id": "12615", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 198." + }, + { + "id": "12616", + "chatId": "999", + "senderId": "703", + "timestamp": "2026-10-04T12:00:00.000Z", + "text": "Unrelated lunch timetable and office stationery reminder 199." + } + ], + "queries": [ + { + "id": "aurora-words", + "split": "dev", + "category": "words", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora rollout rollback", + "legacy": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-typo", + "split": "dev", + "category": "typo", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora rolluot rollback", + "legacy": "aurora rolluot rollback", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-beginnings", + "split": "dev", + "category": "beginnings", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora roll rollback", + "legacy": "aurora roll rollback", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-phrase", + "split": "dev", + "category": "phrase", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "\"aurora rollout rollback\"", + "legacy": "\"aurora rollout rollback\"", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-from", + "split": "dev", + "category": "from", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora rollout from:700", + "legacy": "aurora rollout from:700", + "relevant": { + "10001": 2 + } + }, + { + "id": "aurora-chat", + "split": "dev", + "category": "chat", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora rollout chat:101", + "legacy": "aurora rollout chat:101", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-date", + "split": "dev", + "category": "date", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora rollout date:2026-10-02", + "legacy": "aurora rollout after:2026-10-02 before:2026-10-03", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-and", + "split": "dev", + "category": "and", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora AND rollout AND rollback", + "legacy": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-or", + "split": "dev", + "category": "or", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora rollout OR rollback", + "legacy": "aurora rollout OR rollback", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-paraphrase", + "split": "dev", + "category": "paraphrase", + "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", + "lucene": "aurora deployment reversal", + "legacy": "aurora deployment reversal", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-absent", + "split": "dev", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "aurora quantum unicorn", + "legacy": "aurora quantum unicorn", + "relevant": {} + }, + { + "id": "aurora-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "aurora rollout rollback purple", + "legacy": "aurora rollout rollback purple", + "relevant": {} + }, + { + "id": "cedar-words", + "split": "dev", + "category": "words", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar backup retention", + "legacy": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-typo", + "split": "dev", + "category": "typo", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar bakcup retention", + "legacy": "cedar bakcup retention", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-beginnings", + "split": "dev", + "category": "beginnings", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar back retention", + "legacy": "cedar back retention", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-phrase", + "split": "dev", + "category": "phrase", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "\"cedar backup retention\"", + "legacy": "\"cedar backup retention\"", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-from", + "split": "dev", + "category": "from", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar backup from:700", + "legacy": "cedar backup from:700", + "relevant": { + "10303": 2 + } + }, + { + "id": "cedar-chat", + "split": "dev", + "category": "chat", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar backup chat:102", + "legacy": "cedar backup chat:102", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-date", + "split": "dev", + "category": "date", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar backup date:2026-10-02", + "legacy": "cedar backup after:2026-10-02 before:2026-10-03", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-and", + "split": "dev", + "category": "and", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar AND backup AND retention", + "legacy": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-or", + "split": "dev", + "category": "or", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar backup OR retention", + "legacy": "cedar backup OR retention", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-paraphrase", + "split": "dev", + "category": "paraphrase", + "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", + "lucene": "cedar snapshot lifetime", + "legacy": "cedar snapshot lifetime", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-absent", + "split": "dev", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "cedar quantum unicorn", + "legacy": "cedar quantum unicorn", + "relevant": {} + }, + { + "id": "cedar-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "cedar backup retention purple", + "legacy": "cedar backup retention purple", + "relevant": {} + }, + { + "id": "север-words", + "split": "dev", + "category": "words", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север сертификат обновление", + "legacy": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-typo", + "split": "dev", + "category": "typo", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север сертифкат обновление", + "legacy": "север сертифкат обновление", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-beginnings", + "split": "dev", + "category": "beginnings", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север серти обновление", + "legacy": "север серти обновление", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-phrase", + "split": "dev", + "category": "phrase", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "\"север сертификат обновление\"", + "legacy": "\"север сертификат обновление\"", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-from", + "split": "dev", + "category": "from", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север сертификат from:700", + "legacy": "север сертификат from:700", + "relevant": { + "10605": 2 + } + }, + { + "id": "север-chat", + "split": "dev", + "category": "chat", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север сертификат chat:103", + "legacy": "север сертификат chat:103", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-date", + "split": "dev", + "category": "date", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север сертификат date:2026-10-02", + "legacy": "север сертификат after:2026-10-02 before:2026-10-03", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-and", + "split": "dev", + "category": "and", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север AND сертификат AND обновление", + "legacy": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-or", + "split": "dev", + "category": "or", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север сертификат OR обновление", + "legacy": "север сертификат OR обновление", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-paraphrase", + "split": "dev", + "category": "paraphrase", + "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", + "lucene": "север продление TLS", + "legacy": "север продление TLS", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-absent", + "split": "dev", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "север quantum unicorn", + "legacy": "север quantum unicorn", + "relevant": {} + }, + { + "id": "север-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "север сертификат обновление purple", + "legacy": "север сертификат обновление purple", + "relevant": {} + }, + { + "id": "lumen-words", + "split": "dev", + "category": "words", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen invoice refund", + "legacy": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-typo", + "split": "dev", + "category": "typo", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen invocie refund", + "legacy": "lumen invocie refund", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-beginnings", + "split": "dev", + "category": "beginnings", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen invo refund", + "legacy": "lumen invo refund", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-phrase", + "split": "dev", + "category": "phrase", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "\"lumen invoice refund\"", + "legacy": "\"lumen invoice refund\"", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-from", + "split": "dev", + "category": "from", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen invoice from:700", + "legacy": "lumen invoice from:700", + "relevant": { + "10907": 2 + } + }, + { + "id": "lumen-chat", + "split": "dev", + "category": "chat", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen invoice chat:104", + "legacy": "lumen invoice chat:104", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-date", + "split": "dev", + "category": "date", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen invoice date:2026-10-02", + "legacy": "lumen invoice after:2026-10-02 before:2026-10-03", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-and", + "split": "dev", + "category": "and", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen AND invoice AND refund", + "legacy": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-or", + "split": "dev", + "category": "or", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen invoice OR refund", + "legacy": "lumen invoice OR refund", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-paraphrase", + "split": "dev", + "category": "paraphrase", + "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", + "lucene": "lumen payment reimbursement", + "legacy": "lumen payment reimbursement", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-absent", + "split": "dev", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "lumen quantum unicorn", + "legacy": "lumen quantum unicorn", + "relevant": {} + }, + { + "id": "lumen-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "lumen invoice refund purple", + "legacy": "lumen invoice refund purple", + "relevant": {} + }, + { + "id": "harbor-words", + "split": "test", + "category": "words", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor release freeze", + "legacy": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-typo", + "split": "test", + "category": "typo", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor relesae freeze", + "legacy": "harbor relesae freeze", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-beginnings", + "split": "test", + "category": "beginnings", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor rele freeze", + "legacy": "harbor rele freeze", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-phrase", + "split": "test", + "category": "phrase", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "\"harbor release freeze\"", + "legacy": "\"harbor release freeze\"", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-from", + "split": "test", + "category": "from", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor release from:700", + "legacy": "harbor release from:700", + "relevant": { + "11209": 2 + } + }, + { + "id": "harbor-chat", + "split": "test", + "category": "chat", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor release chat:105", + "legacy": "harbor release chat:105", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-date", + "split": "test", + "category": "date", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor release date:2026-10-02", + "legacy": "harbor release after:2026-10-02 before:2026-10-03", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-and", + "split": "test", + "category": "and", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor AND release AND freeze", + "legacy": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-or", + "split": "test", + "category": "or", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor release OR freeze", + "legacy": "harbor release OR freeze", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-paraphrase", + "split": "test", + "category": "paraphrase", + "intent": "Harbor release freeze decision: stop deployments until the database check passes.", + "lucene": "harbor shipping pause", + "legacy": "harbor shipping pause", + "relevant": { + "11209": 2, + "11210": 1 + } + }, + { + "id": "harbor-absent", + "split": "test", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "harbor quantum unicorn", + "legacy": "harbor quantum unicorn", + "relevant": {} + }, + { + "id": "harbor-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "harbor release freeze purple", + "legacy": "harbor release freeze purple", + "relevant": {} + }, + { + "id": "maple-words", + "split": "test", + "category": "words", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple archive restore", + "legacy": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-typo", + "split": "test", + "category": "typo", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple arhcive restore", + "legacy": "maple arhcive restore", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-beginnings", + "split": "test", + "category": "beginnings", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple arch restore", + "legacy": "maple arch restore", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-phrase", + "split": "test", + "category": "phrase", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "\"maple archive restore\"", + "legacy": "\"maple archive restore\"", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-from", + "split": "test", + "category": "from", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple archive from:700", + "legacy": "maple archive from:700", + "relevant": { + "11511": 2 + } + }, + { + "id": "maple-chat", + "split": "test", + "category": "chat", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple archive chat:106", + "legacy": "maple archive chat:106", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-date", + "split": "test", + "category": "date", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple archive date:2026-10-02", + "legacy": "maple archive after:2026-10-02 before:2026-10-03", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-and", + "split": "test", + "category": "and", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple AND archive AND restore", + "legacy": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-or", + "split": "test", + "category": "or", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple archive OR restore", + "legacy": "maple archive OR restore", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-paraphrase", + "split": "test", + "category": "paraphrase", + "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", + "lucene": "maple recover saved data", + "legacy": "maple recover saved data", + "relevant": { + "11511": 2, + "11512": 1 + } + }, + { + "id": "maple-absent", + "split": "test", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "maple quantum unicorn", + "legacy": "maple quantum unicorn", + "relevant": {} + }, + { + "id": "maple-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "maple archive restore purple", + "legacy": "maple archive restore purple", + "relevant": {} + }, + { + "id": "вектор-words", + "split": "test", + "category": "words", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор миграция отмена", + "legacy": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-typo", + "split": "test", + "category": "typo", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор миграця отмена", + "legacy": "вектор миграця отмена", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-beginnings", + "split": "test", + "category": "beginnings", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор мигр отмена", + "legacy": "вектор мигр отмена", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-phrase", + "split": "test", + "category": "phrase", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "\"вектор миграция отмена\"", + "legacy": "\"вектор миграция отмена\"", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-from", + "split": "test", + "category": "from", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор миграция from:700", + "legacy": "вектор миграция from:700", + "relevant": { + "11813": 2 + } + }, + { + "id": "вектор-chat", + "split": "test", + "category": "chat", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор миграция chat:107", + "legacy": "вектор миграция chat:107", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-date", + "split": "test", + "category": "date", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор миграция date:2026-10-02", + "legacy": "вектор миграция after:2026-10-02 before:2026-10-03", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-and", + "split": "test", + "category": "and", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор AND миграция AND отмена", + "legacy": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-or", + "split": "test", + "category": "or", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор миграция OR отмена", + "legacy": "вектор миграция OR отмена", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-paraphrase", + "split": "test", + "category": "paraphrase", + "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", + "lucene": "вектор возврат прежней схемы", + "legacy": "вектор возврат прежней схемы", + "relevant": { + "11813": 2, + "11814": 1 + } + }, + { + "id": "вектор-absent", + "split": "test", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "вектор quantum unicorn", + "legacy": "вектор quantum unicorn", + "relevant": {} + }, + { + "id": "вектор-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "вектор миграция отмена purple", + "legacy": "вектор миграция отмена purple", + "relevant": {} + }, + { + "id": "prism-words", + "split": "test", + "category": "words", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism renewal discount", + "legacy": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-typo", + "split": "test", + "category": "typo", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism renweal discount", + "legacy": "prism renweal discount", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-beginnings", + "split": "test", + "category": "beginnings", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism rene discount", + "legacy": "prism rene discount", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-phrase", + "split": "test", + "category": "phrase", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "\"prism renewal discount\"", + "legacy": "\"prism renewal discount\"", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-from", + "split": "test", + "category": "from", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism renewal from:700", + "legacy": "prism renewal from:700", + "relevant": { + "12115": 2 + } + }, + { + "id": "prism-chat", + "split": "test", + "category": "chat", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism renewal chat:108", + "legacy": "prism renewal chat:108", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-date", + "split": "test", + "category": "date", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism renewal date:2026-10-02", + "legacy": "prism renewal after:2026-10-02 before:2026-10-03", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-and", + "split": "test", + "category": "and", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism AND renewal AND discount", + "legacy": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-or", + "split": "test", + "category": "or", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism renewal OR discount", + "legacy": "prism renewal OR discount", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-paraphrase", + "split": "test", + "category": "paraphrase", + "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", + "lucene": "prism subscription price reduction", + "legacy": "prism subscription price reduction", + "relevant": { + "12115": 2, + "12116": 1 + } + }, + { + "id": "prism-absent", + "split": "test", + "category": "no-answer-absent", + "intent": "No message answers the requested combination.", + "lucene": "prism quantum unicorn", + "legacy": "prism quantum unicorn", + "relevant": {} + }, + { + "id": "prism-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "intent": "No message answers the requested combination.", + "lucene": "prism renewal discount purple", + "legacy": "prism renewal discount purple", + "relevant": {} + } + ] +} diff --git a/bench/message-search-quality/generate.py b/bench/message-search-quality/generate.py new file mode 100644 index 000000000..12a7b26e9 --- /dev/null +++ b/bench/message-search-quality/generate.py @@ -0,0 +1,60 @@ +import json +from pathlib import Path + +# Fixed invented topics; the emitted corpus, rather than a random seed, is the evaluation input. +families = [ + ('dev', 'aurora', 'rollout', 'rollback', 'Aurora rollout rollback decision: disable the canary and restore build 17.', 'The Aurora rollout rollback owner is Mira; the decision is recorded above.', 'rolluot', 'roll', 'deployment reversal'), + ('dev', 'cedar', 'backup', 'retention', 'Cedar backup retention decision: keep daily snapshots for 35 days.', 'The Cedar backup retention owner is Niko; the policy is recorded above.', 'bakcup', 'back', 'snapshot lifetime'), + ('dev', 'север', 'сертификат', 'обновление', 'Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.', 'Север сертификат обновление проверено, владелец Ира.', 'сертифкат', 'серти', 'продление TLS'), + ('dev', 'lumen', 'invoice', 'refund', 'Lumen invoice refund approved: return 480 credits to the original payer.', 'The Lumen invoice refund owner is Eva; approval is recorded above.', 'invocie', 'invo', 'payment reimbursement'), + ('test', 'harbor', 'release', 'freeze', 'Harbor release freeze decision: stop deployments until the database check passes.', 'The Harbor release freeze owner is Oren; the decision is recorded above.', 'relesae', 'rele', 'shipping pause'), + ('test', 'maple', 'archive', 'restore', 'Maple archive restore decision: recover the March snapshot into a separate workspace.', 'The Maple archive restore owner is Ada; the decision is recorded above.', 'arhcive', 'arch', 'recover saved data'), + ('test', 'вектор', 'миграция', 'отмена', 'Вектор миграция отмена: отключить задание и вернуть схему версии 8.', 'Вектор миграция отмена проверена, владелец Лев.', 'миграця', 'мигр', 'возврат прежней схемы'), + ('test', 'prism', 'renewal', 'discount', 'Prism renewal discount approved: apply 15 percent to the annual contract.', 'The Prism renewal discount owner is Tess; approval is recorded above.', 'renweal', 'rene', 'subscription price reduction'), +] +messages, queries = [], [] +chats = [{'id': str(i), 'title': f'Synthetic {name} decisions'} for i, (_, name, *_) in enumerate(families, 101)] +chats.append({'id': '999', 'title': 'Synthetic busy operations'}) +next_id = 10000 + +def add(chat, sender, text, day=2): + global next_id + next_id += 1 + mid = str(next_id) + messages.append(dict(id=mid, chatId=chat, senderId=sender, timestamp=f'2026-10-{day:02d}T12:00:00.000Z', text=text)) + return mid + +for i, (split, name, word, action, answer, support, typo, prefix, meaning) in enumerate(families, 101): + chat = str(i) + a = add(chat, '700', answer) + b = add(chat, '701', support) + for j in range(300): + templates = [ + f'{name} {word} {action}: agenda only; no decision yet. Thread {j}.', + f'{name} {word} {action}? Please add this question to the meeting. Thread {j}.', + f'Checklist heading: {name} {word} {action}. This empty template supplies no policy. Thread {j}.', + f'{name} status digest. {word} is a document tag. {action} is a separate training exercise. No answer. Thread {j}.', + f'Old {name} {word} {action} proposal rejected; do not treat this as the agreed decision. Thread {j}.', + ] + add('999' if j < 200 else chat, '702', templates[j % len(templates)], 3) + variants = [ + ('words', f'{name} {word} {action}', None), + ('typo', f'{name} {typo} {action}', None), + ('beginnings', f'{name} {prefix} {action}', None), + ('phrase', f'"{name} {word} {action}"', None), + ('from', f'{name} {word} from:700', None), + ('chat', f'{name} {word} chat:{chat}', None), + ('date', f'{name} {word} date:2026-10-02', f'{name} {word} after:2026-10-02 before:2026-10-03'), + ('and', f'{name} AND {word} AND {action}', f'{name} {word} {action}'), + ('or', f'{name} {word} OR {action}', None), + ('paraphrase', f'{name} {meaning}', None), + ] + for category, text, legacy in variants: + relevant = {a: 2, b: 1} + if category == 'from': relevant = {a: 2} + queries.append(dict(id=f'{name}-{category}', split=split, category=category, intent=answer, lucene=text, legacy=legacy or text, relevant=relevant)) + for category, text in [('absent', f'{name} quantum unicorn'), ('near-miss', f'{name} {word} {action} purple')]: + queries.append(dict(id=f'{name}-{category}', split=split, category=f'no-answer-{category}', intent='No message answers the requested combination.', lucene=text, legacy=text, relevant={})) + +for j in range(200): add('999', '703', f'Unrelated lunch timetable and office stationery reminder {j}.', 4) +Path(__file__).with_name('corpus.json').write_text(json.dumps(dict(chats=chats, messages=messages, queries=queries), ensure_ascii=False, indent=2) + '\n') diff --git a/bench/message-search-quality/run.ts b/bench/message-search-quality/run.ts new file mode 100644 index 000000000..37482ed7d --- /dev/null +++ b/bench/message-search-quality/run.ts @@ -0,0 +1,130 @@ +import assert from "node:assert/strict" +import { createHash } from "node:crypto" +import { mkdtempSync, readFileSync } from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { fileURLToPath } from "node:url" +import type { Message } from "../../dist/domain/models.js" +import { searchStore } from "../../dist/services/messages.js" +import { openStore } from "../../dist/store/store.js" + +type Mode = "lucene" | "legacy" +type Query = { + id: string + split: "dev" | "test" + category: string + intent: string + lucene: string + legacy: string + relevant: Record +} +type Corpus = { + chats: { id: string; title: string }[] + messages: Pick[] + queries: Query[] +} +type Row = { + id: string + split: Query["split"] + category: string + mode: Mode + text: string + relevant: Record + hits: { id: string; chatId: string; match: string | null; score: number | null }[] + corrections: { from: string; to: string[] }[] + recallAt10: number | null + mrr: number | null + ndcgAt10: number | null + noAnswerFalseHits: number +} + +const corpusBytes = readFileSync(new URL("./corpus.json", import.meta.url)) +const corpus = JSON.parse(corpusBytes.toString()) as Corpus +const ids = new Set(corpus.messages.map(({ id }) => id)) +assert.equal(ids.size, corpus.messages.length) +assert.equal(new Set(corpus.queries.map(({ id }) => id)).size, corpus.queries.length) +for (const q of corpus.queries) { + for (const [id, grade] of Object.entries(q.relevant)) { + assert.ok(ids.has(id), `unknown relevance label ${id}`) + assert.ok(grade === 1 || grade === 2) + } +} +const root = mkdtempSync(join(tmpdir(), "message-search-quality-")) +const path = join(root, "quality.db") +// Set this before opening any store; the explicit path remains the authority. +process.env.TZ = "UTC" +process.env.MESSAGING_STORE = path +process.env.CLI_COMMON_CACHE_DIR = join(root, "empty-cache") +const account = { provider: "synthetic", account: "500" } +const store = await openStore({ path }) +const rows: Row[] = [] +try { + await store.saveChats(account, corpus.chats.map((chat) => ({ + ...chat, kind: "group", unreadCount: 0, lastMessageAt: "2026-10-04T12:00:00.000Z", participantsCount: 100, + }))) + for (const chat of corpus.chats) { + await store.saveMessages(account, chat.id, corpus.messages.filter((m) => m.chatId === chat.id).map((m): Message => ({ + ...m, senderName: `Synthetic ${m.senderId}`, editedAt: null, outgoing: false, attachments: [], + replyTo: null, forwardedFrom: null, reactions: null, + })), { via: "synthetic" }) + } + await store.fillSearchIndex({}) + await store.fillStems({}) + assert.equal((await store.searchIndexState())?.ready, true) + for (const q of corpus.queries) { + for (const mode of ["lucene", "legacy"] as const) { + const found = await searchStore(store, account, { + text: q[mode], language: mode, limit: 10, + ...(mode === "lucene" ? { timezone: "UTC" } : {}), + }) + assert.ok(found.wordsReady) + if (mode === "lucene") assert.ok(found.stemsReady) + const hits = found.items.map((h) => ({ id: h.id, chatId: h.chatId, match: h.match ?? null, score: h.score ?? null })) + assert.equal(new Set(hits.map((h) => h.id)).size, hits.length) + assert.ok(hits.every((h) => ids.has(h.id))) + const grades = Object.values(q.relevant) + const relevant = hits.filter((h) => q.relevant[h.id]).length + const first = hits.findIndex((h) => q.relevant[h.id]) + const dcg = (g: number[]) => g.reduce((sum, grade, i) => sum + (2 ** grade - 1) / Math.log2(i + 2), 0) + const ideal = dcg([...grades].sort((a, b) => b - a).slice(0, 10)) + rows.push({ + id: q.id, split: q.split, category: q.category, mode, text: q[mode], relevant: q.relevant, hits, + corrections: found.corrections, + recallAt10: grades.length ? relevant / grades.length : null, + mrr: grades.length ? first < 0 ? 0 : 1 / (first + 1) : null, + ndcgAt10: grades.length ? dcg(hits.map((h) => q.relevant[h.id] ?? 0)) / ideal : null, + noAnswerFalseHits: grades.length ? 0 : hits.length, + }) + } + } + assert.equal(rows.length, corpus.queries.length * 2) + const summarize = (selected: Row[]) => { + const answerable = selected.filter((r) => r.recallAt10 !== null) + const negatives = selected.filter((r) => r.recallAt10 === null) + const mean = (key: "recallAt10" | "mrr" | "ndcgAt10") => answerable.length + ? answerable.reduce((sum, r) => sum + (r[key] ?? 0), 0) / answerable.length : null + return { + queries: selected.length, answerable: answerable.length, noAnswerQueries: negatives.length, + recallAt10: mean("recallAt10"), mrr: mean("mrr"), ndcgAt10: mean("ndcgAt10"), + noAnswerFalseHits: negatives.reduce((sum, r) => sum + r.noAnswerFalseHits, 0), + noAnswerQueryHitRate: negatives.length ? negatives.filter((r) => r.hits.length > 0).length / negatives.length : null, + } + } + const digest = (bytes: Buffer) => createHash("sha256").update(bytes).digest("hex") + const productionFiles = ["services/messages.js", "services/messages-search.js", "search/search.js", "search/query.js", + "search/correct.js", "search/stem.js", "search/lucene/parser.js", "store/sqlite/lucene.js", "store/sqlite/words.js"] + console.log(JSON.stringify({ + schemaVersion: 1, k: 10, node: process.version, timezone: "UTC", storePath: path, + corpusSha256: digest(corpusBytes), runnerSha256: digest(readFileSync(fileURLToPath(import.meta.url))), + productionSha256: Object.fromEntries(productionFiles.map((file) => [file, digest(readFileSync(new URL(`../../dist/${file}`, import.meta.url)))])), + messages: corpus.messages.length, queries: corpus.queries.length, + metrics: Object.fromEntries(["dev", "test"].map((split) => [split, Object.fromEntries(["lucene", "legacy"].map((mode) => { + const selected = rows.filter((r) => r.split === split && r.mode === mode) + return [mode, { ...summarize(selected), categories: Object.fromEntries([...new Set(selected.map((r) => r.category))] + .map((category) => [category, summarize(selected.filter((r) => r.category === category))])) }] + }))])), + rankings: rows, + }, null, 2)) +} finally { + await store.close() +} diff --git a/bench/message-search-quality/tsconfig.json b/bench/message-search-quality/tsconfig.json new file mode 100644 index 000000000..86ade7582 --- /dev/null +++ b/bench/message-search-quality/tsconfig.json @@ -0,0 +1,5 @@ +{ + "extends": "@leemour/cli-core/tsconfig.base.json", + "compilerOptions": { "noEmit": true, "composite": false, "declaration": false, "declarationMap": false }, + "include": ["*.ts"] +} diff --git a/docs/dev/combined-search.md b/docs/dev/combined-search.md new file mode 100644 index 000000000..ecfc570a8 --- /dev/null +++ b/docs/dev/combined-search.md @@ -0,0 +1,183 @@ +# One message search: proposal for approval + +Status: benchmark and plan only. No production search behavior has changed. Owner approval is +required before implementation. The proposed first release combines lexical matchers under the +Lucene grammar; meaning search follows only after a separate message-level evaluation. + +## Evidence and acceptance targets + +The [message benchmark](../../bench/message-search-quality/README.md) uses 2,616 synthetic messages, +96 queries, ready word/stem indexes, a temporary store and the current production `searchStore`. +Its [raw baseline](../../bench/message-search-quality/baseline.json) contains every query in both modes. +Four topic families per split give 40 answerable and eight no-answer queries each; there is no +real account, model download or network use. Reproduce with the benchmark README's three commands. + +| Split | Search | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | +|---|---|---|---|---|---| +| dev | strict Lucene | 0.313 | 0.325 | 0.287 | 0 | +| dev | legacy chain | 0.388 | 0.425 | 0.354 | 80 | +| held-out | strict Lucene | 0.263 | 0.325 | 0.219 | 0 | +| held-out | legacy chain | 0.338 | 0.450 | 0.299 | 80 | + +Strict search has zero recall on typos and unfinished words in both splits. Legacy's corresponding +held-out recall is only 0.125: it retrieves keyword-heavy distractors ahead of useful messages. +Its broad fallback also returns ten hits on every no-answer request. Sender/date controls reach +full recall; unfiltered keyword/phrase/chat/Boolean categories remain poor. Inspect category and +per-query results alongside aggregates. These invented templates do not establish real-world quality. + +Proposed release gates, fixed before implementation: + +- Dev and held-out recall@10 at least 0.45, MRR@10 at least 0.50, nDCG@10 at least 0.40. All three + must also beat both baseline modes on their respective split. +- Zero no-answer false hits in each split, preserving the stronger strict baseline. +- Sender/date control recall remains 1.0; phrase and explicit Boolean eligibility stay identical + to the strict baseline. In those categories ranking may improve but cannot broaden eligibility. +- Typo and beginnings recall improve above each split's legacy baseline. Report paraphrase misses + separately; semantic retrieval is not a requirement of the lexical release. +- No scope leaks, duplicated locators or unlabelled combined hits. No change to regex eligibility + or `--newest` chronological ordering. Strict results with the same execution AST remain identical. + +These are acceptance targets, not measured promises. If lexical ranking cannot reach them, bring +back the measured limitation and a revised plan for approval. Do not tune on held-out failures or +weaken labels. Freeze the dev-selected configuration before evaluating held-out; a second development +cycle needs a fresh held-out set. Baseline exposure means this holdout is not blinded. + +## Query behavior + +CLI and MCP accept one language, Lucene, and remove the `--language` option. Parse and resolve once +through the existing parser/registry and `prepareLucene`. Keep account authority, field validation, +timezone handling, coverage, context, thread hydration, cancellation and query budgets. + +Forgiving matching applies to bare positive `text` words in an implicit-conjunction query. Exact +field predicates, exclusions, phrases, wildcard/regex predicates and non-text filters remain hard +constraints in every candidate path. Queries containing explicit AND, OR, NOT, required/prohibited +operators, or an externally supplied AST use the strict path for the first release. Preserve lexical +intent from source tokens: the current AST does not distinguish implicit from explicit conjunction, +so inspecting the AST alone is insufficient. Mixed ordinary words plus phrases/filters may expand +only those ordinary words, with all other predicates checked on every candidate. Excluded words +are never corrected or relaxed. Empty/filter-only requests keep their existing order and carry +`match: filters`; do not invent relevance for a request with no text. + +Example: `aurora rolluot chat:101` can find `rollout` and reports the correction, while +`aurora AND rolluot`, `exact:rolluot`, and `"aurora rolluot"` remain literal. A correction is evidence +of a possible spelling, not permission to remove another requested word or filter. + +`--exact` disables every forgiving candidate path. Preserve its existing default-field mapping to +`exact`, including disabling stemming on bare words/quotes. This is stricter than today's default +Lucene `text` field, which includes stems. Callers needing today's stemmed strict behavior can use +explicit `text:` predicates or an AST; the plan does not silently redefine the existing exact flag. +Document this distinction in CLI/MCP discovery and migration examples. + +Keep `--regex` as the explicit JavaScript regex operation, using the internal regex implementation +and its existing account restrictions. It does not require a selectable legacy query language. +Keep the legacy chain internally during compatibility work; bot message search moves onto the +combined path only after its account restrictions and default-request tests pass. + +## Candidates, fusion and ranking + +Run eligible matchers independently so an exact hit cannot suppress useful corrected/prefix hits. +Reuse indexed word, stem, prefix and correction primitives; do not translate arbitrary Lucene ASTs +into legacy WordQuery, which would lose Boolean and field semantics. Build bounded transformed +executions from the resolved query and validate every hard predicate in the same authority scope. +No table or migration is proposed for the lexical release. + +Candidate sources are exact words, stems, word beginnings and corrected whole words/beginnings. +Correct only unknown ordinary words with the existing edit-distance/trigram vocabulary rules. +Each source retains its own rank position; raw word BM25, stem BM25 and timestamp ranks are not +numerically comparable. Collapse duplicates by full message locator, including provider/account. +Exact/stem lists share one words contribution per locator so correlated lists cannot double-vote; +corrected whole/prefix lists similarly share one corrected contribution. Prefix adds a contribution +only when it actually expands a term, avoiding repeated votes for the same exact match. + +Fuse independent lists with reciprocal rank fusion. Start with k=60 as notes/conversations do; +compare k in {20, 60, 100} and per-source candidate depth in {50, 100, 300, 500} on dev only. +The corpus intentionally has 300 distractors per topic, so depth 50 must not be assumed sufficient. +Choose the smallest configuration meeting the dev gates; break ties by higher nDCG, then lower +measured latency. Cache correction vocabulary reads once per query; concurrent promises do not +make a synchronous SQLite driver compute in parallel. + +Rerank the bounded union against the complete query. Evaluate term coverage, literal-vs-stem-vs- +correction strength, minimum token span, phrase adjacency and normalized RRF; compare each signal +by dev ablation instead of committing unmeasured weights. Never treat recency as answer quality. +Use deterministic ties: fused rank, timestamp, then full locator. Simple lexical signals cannot +reliably distinguish a decision from an unanswered question with identical words; the benchmark +includes that limitation deliberately. Do not add corpus-specific answer words or project names. + +Compare no diversity cap with a soft cap of three or five hits per chat in the first ten results. +Apply it after reranking, backfilling deferred hits if no other chats qualify; disable it when the +request scopes to one chat. Expand it proportionally on larger pages and evaluate pagination so +hits cannot be silently lost. A conversation cap requires conversation membership and extra work; +defer it with the meaning phase. Adopt the cap only if dev improves without reducing control recall. + +Drop automatic any-word and raw-substring candidates from the combined lexical default. The legacy +benchmark's no-answer noise makes their inclusion unjustified. Existing explicit Lucene wildcard, +regex and OR expressions provide deliberate broader retrieval. The SDK compatibility chain keeps +its old weak steps until the breaking transition; the new default does not pad empty results. + +Every combined hit carries the strongest `match` label (`words`, `beginnings`, `corrected`, or +`filters`), a `matches` list of all contributing labels, and a finite positive `score`. Stems use +`words` and existing query stemming metadata. Scores express ordering within this query, not a +probability or a cross-query confidence threshold. Return only corrections that contributed an +eligible hit. Preserve readiness/coverage even on empty results. If the word index is not ready, +use the current Lucene readiness error, rather than fall back to an unlabelled substring result. +Strict/regex paths keep their current result shapes unless an additive label is needed for consumers. + +Candidate truncation must be visible in query metadata (`candidateDepth`, `truncated`) and +`hasMore`; a bounded shortlist is not proof the archive has no more matches. Define deterministic +paging over that shortlist before enabling it in `search all`, saved replay and MCP. Test limits +1/10/50/1000 and `--newest`; chronological requests sort eligible combined candidates by timestamp +and report truncation rather than claim to have scanned the full archive. Measure warmed query +p50/p95 on this corpus and a generated 100k-message isolated archive. Proposed latency gate: +p95 <= 250 ms and <= 3x strict for ordinary lexical queries; retain existing execution budgets. + +## Meaning phase + +Do not ship meaning in the first release. Conversation-chunk vectors do not supply independent +message ranks, and the conversation e5-small floor of 0.80 is not a validated message threshold. +A later measured proposal must map candidate chunks only to their member messages, rerank each +message against the query, bound contributions per chunk/conversation, enforce every hard filter, +and preserve lexical-only behavior when the local model/index is unavailable. It must never +install a model or embed the archive as a search side effect. Add a `meaning` label through an +explicit type/discovery update and choose its threshold on a separate dev set with no-answer +controls. Record mapping, freshness, coverage and latency, then seek approval for that release. + +## Compatibility and release order + +This proposal includes the separately agreed breaking window required by the +[stable-export policy](../../README.md#how-often-and-what-may-break). + +1. Build behind an internal entry point after approval. Keep default `MessagesService.search` and + `searchStore` legacy semantics, and explicit language requests, until the coordinated breaking + release. Test old SDK requests against the unchanged chain. Add new combined tests and benchmark + measurement without replacing the committed baseline. +2. Adapt saved replay before switching the default. Keep stored legacy originals and convert on + read into a versioned Lucene execution with preserved filters/exclusions and bare positive words + eligible for forgiveness. The current `migrateLegacyQuery` quotes every term and adds explicit + AND; reusing it unchanged would disable the intended expansion. Extend a dedicated conversion + with provenance and warnings; evaluate rolling relative-date filters at replay time, preserving their + original moving window, and report UTC normalization when the original timezone is unavailable. + Store regex records as regex. No destructive rewrite of saved searches or history. A changed + replay keeps `migratedFrom: legacy` metadata; newly saved records use Lucene. Any conversion that + cannot preserve hard eligibility fails with the saved id and an actionable preview. +3. In one cli-messaging breaking release, switch no-language SDK requests to combined Lucene, + remove CLI/MCP language selection, and retain the deprecated SDK language field for one documented + transition release. Explicit SDK `language: lucene` keeps strict semantics during that transition; + `language: legacy` continues the old chain. A later breaking release can remove that field. + Old SDK users preserve discovery temporarily with explicit legacy or migrate to combined; + scripts use `--exact` or explicit `text:`/AST for strict stemmed matching. +4. Update parser error alternatives, registry-generated tables/recipes, query specification, search + schemas, rendering, `search all` scoring, bot paths, saved replay, command reference and skills. + Put user-visible changes and migration instructions under `Changed — may break callers`. + Run `pnpm lint`, `pnpm typecheck`, `pnpm test:coverage`, `pnpm docs:check`, the benchmark's own + TypeScript check, dev ablations and the frozen held-out evaluation. Record the quality/latency + report and exact production hashes. Never touch the real store or `src/replies/**`. +5. Release cli-messaging first, then create draft PRs in tg-cli and max-cli pinning its exact published + version, updating generated `docs/commands.md`, skills and breaking changelog entries. Avoid other + consumer `docs/*.md` prose owned by `docs/reader-standards`. Verify unreleased builds with + `bin/try-messaging` in consumer worktrees; no committed `file:` dependency. Arrange same-day + consumer moves when the breaking shared release lands. No release/publish is part of this plan-only task. + +No migration number is needed now. If saved conversion or later semantic work requires schema +changes, reserve a number first through [coordination](COORDINATION.md#store-migrations). +Approval covers the lexical design, quality gates, deferred meaning phase and coordinated SDK +transition. Any failure to meet those gates returns to a measured proposal before release. From 37cd091628af71b6f6022f85a2932718168ab547 Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 14:06:29 +0200 Subject: [PATCH 02/14] docs(search): include discovery and completion in transition plan --- docs/dev/combined-search.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/dev/combined-search.md b/docs/dev/combined-search.md index ecfc570a8..e4484a53d 100644 --- a/docs/dev/combined-search.md +++ b/docs/dev/combined-search.md @@ -167,6 +167,8 @@ This proposal includes the separately agreed breaking window required by the scripts use `--exact` or explicit `text:`/AST for strict stemmed matching. 4. Update parser error alternatives, registry-generated tables/recipes, query specification, search schemas, rendering, `search all` scoring, bot paths, saved replay, command reference and skills. + Remove language selection from the parity option catalogue and regenerate command discovery, + shell completion and reference tables together; account permission flags stay unchanged. Put user-visible changes and migration instructions under `Changed — may break callers`. Run `pnpm lint`, `pnpm typecheck`, `pnpm test:coverage`, `pnpm docs:check`, the benchmark's own TypeScript check, dev ablations and the frozen held-out evaluation. Record the quality/latency From 0c53a4b06e7e112d02ffd9958344c5236fe84e0a Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 14:30:48 +0200 Subject: [PATCH 03/14] feat(search): evaluate combined lexical ranking behind internal entry point --- bench/message-search-quality/README.md | 23 + .../message-search-quality/combined-dev.json | 3723 +++++++++++++++++ bench/message-search-quality/combined.ts | 184 + docs/dev/combined-search.md | 55 +- src/search/lucene/parser.ts | 15 + src/services/messages-combined.test.ts | 341 ++ src/services/messages-combined.ts | 314 ++ 7 files changed, 4651 insertions(+), 4 deletions(-) create mode 100644 bench/message-search-quality/combined-dev.json create mode 100644 bench/message-search-quality/combined.ts create mode 100644 src/services/messages-combined.test.ts create mode 100644 src/services/messages-combined.ts diff --git a/bench/message-search-quality/README.md b/bench/message-search-quality/README.md index b9ed0df41..206be31ac 100644 --- a/bench/message-search-quality/README.md +++ b/bench/message-search-quality/README.md @@ -75,3 +75,26 @@ The strict matcher misses typos and word beginnings. Legacy recovers some, but i list prevents other matchers from contributing; broad any-word fallback fills no-answer pages. Both can bury useful messages beneath exact keyword matches. Sender/date filters are easy controls and achieve full recall; category reports prevent them from concealing poor unfiltered rankings. + +## Combined lexical experiment + +After building, run the separate dev evaluation: + +```sh +pnpm exec tsc -p bench/message-search-quality/tsconfig.json +node bench/message-search-quality/combined.ts > bench/message-search-quality/combined-dev.json +``` + +It calls the internal production matcher directly without changing public `searchStore` defaults. +All 180 combinations of candidate depth, RRF constant, lexical reranking signal and chat cap are +measured on dev only. All returned rankings are scored against the unchanged corpus labels. +Explicit AND/OR/phrase query ids must match the strict production top ten exactly. The runner +never supplies query `intent` or relevance labels to the matcher. + +Every tested configuration has dev recall@10 0.3625, MRR@10 0.375 and nDCG@10 0.3264, with zero +no-answer false hits. None meets the approved 0.45/0.50/0.40 gates. [combined-dev.json](combined-dev.json) +records each configuration, measured timing and hashes, and the diagnostic dev rankings. No +combined held-out result is produced when dev fails. The original baseline is preserved. +The [revised proposal](../../docs/dev/combined-search.md#measured-lexical-experiment-and-revised-proposal) +requests a semantic evaluation before public adoption. Timings are from this small corpus on a +shared machine; they are not release-scale latency evidence. diff --git a/bench/message-search-quality/combined-dev.json b/bench/message-search-quality/combined-dev.json new file mode 100644 index 000000000..7832edb41 --- /dev/null +++ b/bench/message-search-quality/combined-dev.json @@ -0,0 +1,3723 @@ +{ + "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", + "productionSha256": { + "services/messages-combined.js": "d10aeb5f7049f615503d59c37df109bcd284f53b5cfceaac0151f2d7e2ca60e9", + "services/messages-search.js": "35eb4ed1dcc002a80985327ffa9f6bd25a5ee6b1a31f3024203a38b6605c7d2a", + "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", + "search/correct.js": "54e5089f217840dd9e27e6e89c29486c2d8704850af16dafd1b8410998066270", + "search/stem.js": "84475baa966a17595c480015668dc77da60409a00c391aa9ca6a718433386007", + "store/normalize.js": "2c59c9a7c7486b37754b639d26d036c20054aff2c18470ba0524115d9da24de0", + "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964" + }, + "runnerSha256": "b71dd53f4ba99a5d690a313569b048c0bc801d27fa29576e74ec0b58fa50ee44", + "configurations": [ + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0989270000000033, + "p95Ms": 5.586588999999947 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.820695999999998, + "p95Ms": 4.736476000000039 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.999600999999984, + "p95Ms": 8.122931999999992 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0147190000000137, + "p95Ms": 5.35894200000007 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0481620000000476, + "p95Ms": 5.102151000000049 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.9879390000000967, + "p95Ms": 4.4711580000000595 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1766829999999118, + "p95Ms": 5.639487000000145 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.120296999999937, + "p95Ms": 4.803971999999931 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1359469999999874, + "p95Ms": 4.584931999999981 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.931483000000071, + "p95Ms": 4.990330999999969 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.067167999999583, + "p95Ms": 5.065161999999873 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.195528000000195, + "p95Ms": 5.348733000000266 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.9227169999999205, + "p95Ms": 4.651435999999649 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0846299999998337, + "p95Ms": 4.821284000000105 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.550333000000137, + "p95Ms": 6.034327000000303 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.6591629999998077, + "p95Ms": 4.640214999999898 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.909552000000076, + "p95Ms": 5.487913999999819 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.993449999999939, + "p95Ms": 5.075561000000107 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.9848830000000817, + "p95Ms": 4.624846000000161 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1461449999997058, + "p95Ms": 5.130724999999984 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.8890740000001642, + "p95Ms": 4.975163000000066 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1842569999998886, + "p95Ms": 4.559173000000101 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.094759000000522, + "p95Ms": 4.604597999999896 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.8531670000002123, + "p95Ms": 4.757614999999532 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1271700000006604, + "p95Ms": 4.51660400000037 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2620529999994687, + "p95Ms": 5.642542999999932 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.9432660000002215, + "p95Ms": 5.173072999999931 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0052319999995234, + "p95Ms": 4.6353769999996075 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2169080000003305, + "p95Ms": 4.748657999999523 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 60, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.071444999999585, + "p95Ms": 4.6720350000005055 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.825594999999339, + "p95Ms": 4.481768000000557 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.948145000000295, + "p95Ms": 5.107079999999769 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.072377000000415, + "p95Ms": 4.858803999999509 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.141276999999718, + "p95Ms": 4.986815000000206 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1549220000006244, + "p95Ms": 4.732707999999548 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.8966380000001664, + "p95Ms": 4.789594999999281 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.083798999999999, + "p95Ms": 4.598597000000154 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1582180000004882, + "p95Ms": 4.535899999999856 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.871831999999813, + "p95Ms": 4.872200000000703 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1740379999992, + "p95Ms": 4.445510999999897 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1011509999998452, + "p95Ms": 4.676784000000225 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.089318999999705, + "p95Ms": 4.93543899999986 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2209860000002664, + "p95Ms": 5.816338999999971 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0475500000002285, + "p95Ms": 4.833337000000029 + } + }, + { + "options": { + "candidateDepth": 50, + "rrfK": 100, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.9232780000002094, + "p95Ms": 4.709133999999722 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1602720000000772, + "p95Ms": 5.875841000000037 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.964536000001317, + "p95Ms": 5.311302000000069 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.943435999999565, + "p95Ms": 5.634228000000803 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3664279999993596, + "p95Ms": 5.946183000000019 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3577520000017103, + "p95Ms": 5.880669000000125 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4458269999995537, + "p95Ms": 5.950079000000187 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4552650000005087, + "p95Ms": 6.402388000000428 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3393580000010843, + "p95Ms": 5.944709999999759 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.42479800000001, + "p95Ms": 6.081726000000344 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.377369000001636, + "p95Ms": 5.775934000001143 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4498050000001967, + "p95Ms": 5.79357600000003 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3980179999998654, + "p95Ms": 6.328649000000951 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3770779999995284, + "p95Ms": 5.879417000000103 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3765569999995932, + "p95Ms": 5.947375000001557 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 20, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.438093000000663, + "p95Ms": 6.328408000001218 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.979233000000022, + "p95Ms": 5.716100999999981 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.99686500000098, + "p95Ms": 5.263794000000416 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.9873480000005657, + "p95Ms": 5.555910999999469 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.24366800000098, + "p95Ms": 7.042266000000382 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.120136999999886, + "p95Ms": 5.631272000000536 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1539000000011583, + "p95Ms": 7.556660000000193 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2382180000004155, + "p95Ms": 6.014180000000124 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2509419999987585, + "p95Ms": 6.167507000000114 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3813960000006773, + "p95Ms": 6.469952999999805 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.154620999999679, + "p95Ms": 6.012516999999207 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4117029999997612, + "p95Ms": 9.632669999999052 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2852060000004712, + "p95Ms": 6.46590599999945 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2030119999999442, + "p95Ms": 6.237648000000263 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.281227999999828, + "p95Ms": 5.967381999998906 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 60, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2434680000005756, + "p95Ms": 6.523754999998346 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.147037000000637, + "p95Ms": 5.403916000001118 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 2.8970179999996617, + "p95Ms": 6.438083999999435 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.028715000000375, + "p95Ms": 5.09686099999999 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.493426000000909, + "p95Ms": 6.166385000000446 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3336959999996907, + "p95Ms": 6.222479999998541 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.337985000000117, + "p95Ms": 5.702556000000186 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.463880999999674, + "p95Ms": 6.249671000001399 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4468889999989187, + "p95Ms": 5.7435329999989335 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.197732000000542, + "p95Ms": 6.895350999999209 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4615969999995286, + "p95Ms": 10.654305000000022 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3023880000000645, + "p95Ms": 5.465963000000556 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.134103000000323, + "p95Ms": 6.911190000000715 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1532289999995555, + "p95Ms": 5.701062999998612 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2860579999996844, + "p95Ms": 6.078510000001188 + } + }, + { + "options": { + "candidateDepth": 100, + "rrfK": 100, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.207480000000942, + "p95Ms": 7.317231000000902 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.100669999999809, + "p95Ms": 9.202213999997184 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2173079999993206, + "p95Ms": 7.690781000001152 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1045169999997597, + "p95Ms": 9.821302999996988 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3173769999993965, + "p95Ms": 10.891920000001846 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.295234999997774, + "p95Ms": 11.62250899999708 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.147939000002225, + "p95Ms": 9.846882000001642 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.125597000002017, + "p95Ms": 10.109012999997503 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1926320000020496, + "p95Ms": 10.076131999998324 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.145183000000543, + "p95Ms": 10.246551000000181 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1659529999997176, + "p95Ms": 10.46642200000133 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.196890999999596, + "p95Ms": 10.91766799999823 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2881820000002335, + "p95Ms": 11.291237999997975 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.236605000001873, + "p95Ms": 9.723580999998376 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.333716999997705, + "p95Ms": 11.087255999998888 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 20, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.184436999999889, + "p95Ms": 10.429353000003175 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0512780000026396, + "p95Ms": 8.476332999998704 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.172945999998774, + "p95Ms": 7.6514379999971425 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0637710000009974, + "p95Ms": 9.070777000000817 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.162156000002142, + "p95Ms": 10.658501999998407 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.143499999998312, + "p95Ms": 10.011179999997694 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.213631999999052, + "p95Ms": 10.287045999997645 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3347690000009607, + "p95Ms": 10.174446000000898 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2254939999984344, + "p95Ms": 10.048009000001912 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2880409999997937, + "p95Ms": 12.38845399999991 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.256471999997302, + "p95Ms": 12.288046000001486 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.211076999999932, + "p95Ms": 10.37240699999893 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1490610000000743, + "p95Ms": 9.428126999999222 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.7868760000019392, + "p95Ms": 10.107739999999467 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.550893999999971, + "p95Ms": 9.652497000002768 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 60, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0478020000009565, + "p95Ms": 14.126581999997143 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0947699999996985, + "p95Ms": 7.647890999996889 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.140835000001971, + "p95Ms": 7.737259000001359 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.240011999998387, + "p95Ms": 10.344193999997515 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 4.588929999998072, + "p95Ms": 13.008075999998255 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2248130000007222, + "p95Ms": 8.985486999998102 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.169378999999026, + "p95Ms": 9.119418000002042 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.137829999999667, + "p95Ms": 11.149172000001272 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 4.387360999997327, + "p95Ms": 10.366034999999101 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1787359999980254, + "p95Ms": 9.140016999997897 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0379629999988538, + "p95Ms": 12.552051000002393 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1403140000002168, + "p95Ms": 9.01670699999886 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.640871999999945, + "p95Ms": 11.654589000001579 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.325831999998627, + "p95Ms": 9.39201999999932 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3790619999999763, + "p95Ms": 9.77812300000005 + } + }, + { + "options": { + "candidateDepth": 300, + "rrfK": 100, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.178325999997469, + "p95Ms": 10.559667999998055 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1289729999989504, + "p95Ms": 8.623279000003095 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0909520000022894, + "p95Ms": 8.486472000000504 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.165442000001349, + "p95Ms": 8.397134999999253 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.136797999999544, + "p95Ms": 9.770067999997991 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2483370000009018, + "p95Ms": 10.034593999997014 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3170260000006238, + "p95Ms": 11.129775999997946 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3663180000003194, + "p95Ms": 10.832407999998395 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.340158999999403, + "p95Ms": 11.217831000001752 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1947459999973944, + "p95Ms": 10.823953000002803 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.260519999999815, + "p95Ms": 10.102961000000505 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2751270000007935, + "p95Ms": 10.182721000001038 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1247150000017427, + "p95Ms": 10.908601999999519 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2969080000002577, + "p95Ms": 10.397753999997803 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.223972000003414, + "p95Ms": 10.098493000001326 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 20, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.16842700000052, + "p95Ms": 11.122252000001026 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3580820000024687, + "p95Ms": 10.930592000000615 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2287099999994098, + "p95Ms": 8.901921000000584 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.134644000001572, + "p95Ms": 8.8235339999992 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1865209999996296, + "p95Ms": 11.053323000000091 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.318026999997528, + "p95Ms": 10.279723000003287 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.210897000000841, + "p95Ms": 10.048931000001176 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1660120000015013, + "p95Ms": 10.784888999998657 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2241810000014084, + "p95Ms": 10.18936400000166 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2118379999992612, + "p95Ms": 10.870098999999755 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2974390000017593, + "p95Ms": 12.574764000000869 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2604190000020026, + "p95Ms": 10.03881100000217 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.124244999999064, + "p95Ms": 9.719203000000562 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.260980000006384, + "p95Ms": 9.93393400000059 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1301149999999325, + "p95Ms": 10.841444999998203 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 60, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.223489999996673, + "p95Ms": 10.250748999998905 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.157636999996612, + "p95Ms": 8.342072000006738 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "none", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.19188099999883, + "p95Ms": 8.674554999997781 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "none", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.102082999997947, + "p95Ms": 8.305793999999878 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2199129999935394, + "p95Ms": 12.216662999999244 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 4.07511599999998, + "p95Ms": 19.6373969999986 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "coverage", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.1366780000025756, + "p95Ms": 10.087242000001424 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.272711999998137, + "p95Ms": 11.465043999996851 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4137569999948028, + "p95Ms": 9.849736999996821 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "proximity", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4071749999930034, + "p95Ms": 10.754502999996475 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.142929000001459, + "p95Ms": 10.303015999998024 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.5304660000037984, + "p95Ms": 9.76239400000486 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "phrase", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4685190000018338, + "p95Ms": 11.494158000001335 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "all", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.4173440000013215, + "p95Ms": 15.484094999999797 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "all", + "chatCap": 3 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.2240810000002966, + "p95Ms": 10.975125999997545 + } + }, + { + "options": { + "candidateDepth": 500, + "rrfK": 100, + "rerank": "all", + "chatCap": 5 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.3575010000058683, + "p95Ms": 10.992918999996618 + } + } + ], + "devPassed": false, + "chosen": { + "options": { + "candidateDepth": 50, + "rrfK": 20, + "rerank": "none", + "chatCap": 0 + }, + "metrics": { + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0, + "p50Ms": 3.0989270000000033, + "p95Ms": 5.586588999999947 + } + }, + "devRankings": [ + { + "id": "aurora-words", + "category": "words", + "ids": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253" + ], + "elapsedMs": 3.976211000001058, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-typo", + "category": "typo", + "ids": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253" + ], + "elapsedMs": 4.017217000000528, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "ids": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253" + ], + "elapsedMs": 2.7637489999979152, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-phrase", + "category": "phrase", + "ids": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253" + ], + "elapsedMs": 1.8199510000049486, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-from", + "category": "from", + "ids": [ + "10001" + ], + "elapsedMs": 3.2671019999979762, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "aurora-chat", + "category": "chat", + "ids": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253" + ], + "elapsedMs": 3.378069999998843, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-date", + "category": "date", + "ids": [ + "10001", + "10002" + ], + "elapsedMs": 3.196129000003566, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "aurora-and", + "category": "and", + "ids": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253" + ], + "elapsedMs": 1.734269000000495, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-or", + "category": "or", + "ids": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253" + ], + "elapsedMs": 1.7011679999995977, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "ids": [], + "elapsedMs": 2.1651070000007167, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "ids": [], + "elapsedMs": 2.0100359999996726, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "ids": [], + "elapsedMs": 2.2358600000006845, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + { + "id": "cedar-words", + "category": "words", + "ids": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555" + ], + "elapsedMs": 3.978183999999601, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-typo", + "category": "typo", + "ids": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555" + ], + "elapsedMs": 3.918993999999657, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "ids": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555" + ], + "elapsedMs": 2.6834280000039143, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-phrase", + "category": "phrase", + "ids": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555" + ], + "elapsedMs": 1.6508929999981774, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-from", + "category": "from", + "ids": [ + "10303" + ], + "elapsedMs": 3.0946489999987534, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "cedar-chat", + "category": "chat", + "ids": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555" + ], + "elapsedMs": 3.4562460000015562, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-date", + "category": "date", + "ids": [ + "10303", + "10304" + ], + "elapsedMs": 3.1982330000028014, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "cedar-and", + "category": "and", + "ids": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555" + ], + "elapsedMs": 1.737676000004285, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-or", + "category": "or", + "ids": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555" + ], + "elapsedMs": 1.6936129999958212, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "ids": [], + "elapsedMs": 1.9198580000011134, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "ids": [], + "elapsedMs": 1.9002000000036787, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "ids": [], + "elapsedMs": 2.203449000000546, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + { + "id": "север-words", + "category": "words", + "ids": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867" + ], + "elapsedMs": 7.92291600000317, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + { + "id": "север-typo", + "category": "typo", + "ids": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867" + ], + "elapsedMs": 4.4926289999930304, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + { + "id": "север-beginnings", + "category": "beginnings", + "ids": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867" + ], + "elapsedMs": 2.7162700000044424, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + { + "id": "север-phrase", + "category": "phrase", + "ids": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862" + ], + "elapsedMs": 1.6422370000000228, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + { + "id": "север-from", + "category": "from", + "ids": [ + "10605" + ], + "elapsedMs": 3.113615000002028, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "север-chat", + "category": "chat", + "ids": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867" + ], + "elapsedMs": 3.446919000001799, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "север-date", + "category": "date", + "ids": [ + "10605", + "10606" + ], + "elapsedMs": 3.272292000001471, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "север-and", + "category": "and", + "ids": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867" + ], + "elapsedMs": 1.7656690000003437, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + { + "id": "север-or", + "category": "or", + "ids": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867" + ], + "elapsedMs": 1.7154039999950328, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "ids": [], + "elapsedMs": 2.115262999999686, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "ids": [], + "elapsedMs": 1.9881450000029872, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "ids": [], + "elapsedMs": 2.263823000001139, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + { + "id": "lumen-words", + "category": "words", + "ids": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159" + ], + "elapsedMs": 3.875832999998238, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-typo", + "category": "typo", + "ids": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159" + ], + "elapsedMs": 4.720746999999392, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "ids": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159" + ], + "elapsedMs": 2.649865999999747, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-phrase", + "category": "phrase", + "ids": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159" + ], + "elapsedMs": 1.5868439999976545, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-from", + "category": "from", + "ids": [ + "10907" + ], + "elapsedMs": 3.1440010000005714, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + { + "id": "lumen-chat", + "category": "chat", + "ids": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159" + ], + "elapsedMs": 3.2406519999931334, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-date", + "category": "date", + "ids": [ + "10908", + "10907" + ], + "elapsedMs": 3.1149070000028587, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + { + "id": "lumen-and", + "category": "and", + "ids": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159" + ], + "elapsedMs": 1.6945750000013504, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-or", + "category": "or", + "ids": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159" + ], + "elapsedMs": 1.7035219999961555, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "ids": [], + "elapsedMs": 2.048427999994601, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "ids": [], + "elapsedMs": 1.8857040000002598, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "ids": [], + "elapsedMs": 2.1373449999955483, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + ], + "heldOut": null +} diff --git a/bench/message-search-quality/combined.ts b/bench/message-search-quality/combined.ts new file mode 100644 index 000000000..3b97d0b67 --- /dev/null +++ b/bench/message-search-quality/combined.ts @@ -0,0 +1,184 @@ +import assert from "node:assert/strict" +import { createHash } from "node:crypto" +import { mkdtempSync, readFileSync } from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { performance } from "node:perf_hooks" +import type { Message } from "../../dist/domain/models.js" +import { searchCombined, type CombinedOptions } from "../../dist/services/messages-combined.js" +import { searchStore } from "../../dist/services/messages.js" +import { openStore } from "../../dist/store/store.js" + +type Query = { id: string; split: "dev" | "test"; category: string; lucene: string; relevant: Record } +type Corpus = { + chats: { id: string; title: string }[] + messages: Pick[] + queries: Query[] +} +type Row = { + id: string + category: string + recallAt10: number | null + mrr: number | null + ndcgAt10: number | null + falseHits: number + ids: string[] + elapsedMs: number +} +const bytes = readFileSync(new URL("./corpus.json", import.meta.url)) +const corpus = JSON.parse(bytes.toString()) as Corpus +const root = mkdtempSync(join(tmpdir(), "combined-quality-")) +process.env.TZ = "UTC" +process.env.MESSAGING_STORE = join(root, "quality.db") +process.env.CLI_COMMON_CACHE_DIR = join(root, "empty-cache") +const store = await openStore({ path: process.env.MESSAGING_STORE }) +const account = { provider: "synthetic", account: "500" } +const report = (rows: Row[]) => { + const answerable = rows.filter((r) => r.recallAt10 !== null) + const average = (key: "recallAt10" | "mrr" | "ndcgAt10") => + answerable.reduce((s, r) => s + (r[key] ?? 0), 0) / answerable.length + const times = rows.map((r) => r.elapsedMs).sort((a, b) => a - b) + return { + recallAt10: average("recallAt10"), + mrr: average("mrr"), + ndcgAt10: average("ndcgAt10"), + noAnswerFalseHits: rows.reduce((s, r) => s + r.falseHits, 0), + p50Ms: times[Math.floor(times.length * 0.5)], + p95Ms: times[Math.floor(times.length * 0.95)], + } +} +const configurations: CombinedOptions[] = [] +for (const candidateDepth of [50, 100, 300, 500]) + for (const rrfK of [20, 60, 100]) + for (const rerank of ["none", "coverage", "proximity", "phrase", "all"] as const) + for (const chatCap of [0, 3, 5]) configurations.push({ candidateDepth, rrfK, rerank, chatCap }) +try { + await store.saveChats( + account, + corpus.chats.map((c) => ({ ...c, kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 100 })), + ) + for (const chat of corpus.chats) + await store.saveMessages( + account, + chat.id, + corpus.messages + .filter((m) => m.chatId === chat.id) + .map( + (m): Message => ({ + ...m, + senderName: `Synthetic ${m.senderId}`, + editedAt: null, + outgoing: false, + attachments: [], + replyTo: null, + forwardedFrom: null, + reactions: null, + }), + ), + { via: "synthetic" }, + ) + await store.fillSearchIndex({}) + await store.fillStems({}) + const run = async (options: CombinedOptions, queries: Query[]) => { + const rows: Row[] = [] + for (const q of queries) { + const start = performance.now() + const found = await searchCombined(store, account, { text: q.lucene, limit: 10, timezone: "UTC" }, {}, options) + const elapsedMs = performance.now() - start + const ids = found.items.map((h) => h.id) + assert.equal(new Set(ids).size, ids.length) + const grades = Object.values(q.relevant) + const first = ids.findIndex((id) => q.relevant[id]) + const dcg = (g: number[]) => g.reduce((s, v, i) => s + (2 ** v - 1) / Math.log2(i + 2), 0) + rows.push({ + id: q.id, + category: q.category, + ids, + elapsedMs, + recallAt10: grades.length ? ids.filter((id) => q.relevant[id]).length / grades.length : null, + mrr: grades.length ? (first < 0 ? 0 : 1 / (first + 1)) : null, + ndcgAt10: grades.length + ? dcg(ids.map((id) => q.relevant[id] ?? 0)) / dcg([...grades].sort((a, b) => b - a).slice(0, 10)) + : null, + falseHits: grades.length ? 0 : ids.length, + }) + if (["phrase", "and", "or"].includes(q.category)) { + const strict = await searchStore(store, account, { + text: q.lucene, + language: "lucene", + timezone: "UTC", + limit: 10, + }) + assert.deepEqual( + ids, + strict.items.map((h) => h.id), + `strict eligibility/order: ${q.id}`, + ) + } + } + return rows + } + const dev = corpus.queries.filter((q) => q.split === "dev") + const results = [] + for (const options of configurations) { + const rows = await run(options, dev) + results.push({ options, metrics: report(rows) }) + } + const meetsGates = (m: ReturnType) => + m.recallAt10 >= 0.45 && m.mrr >= 0.5 && m.ndcgAt10 >= 0.4 && m.noAnswerFalseHits === 0 + const eligible = results.filter((r) => meetsGates(r.metrics)) + const chosen = [...(eligible.length ? eligible : results)].sort((a, b) => + eligible.length + ? a.options.candidateDepth - b.options.candidateDepth || + b.metrics.ndcgAt10 - a.metrics.ndcgAt10 || + (a.metrics.p95Ms ?? 0) - (b.metrics.p95Ms ?? 0) + : b.metrics.ndcgAt10 - a.metrics.ndcgAt10 || + b.metrics.recallAt10 - a.metrics.recallAt10 || + a.options.candidateDepth - b.options.candidateDepth, + )[0] + assert.ok(chosen) + const chosenRows = await run(chosen.options, dev) + // No held-out evaluation until a dev configuration meets the approved release gates. + console.log( + JSON.stringify( + { + corpusSha256: createHash("sha256").update(bytes).digest("hex"), + productionSha256: Object.fromEntries( + [ + "services/messages-combined.js", + "services/messages-search.js", + "search/lucene/parser.js", + "search/correct.js", + "search/stem.js", + "store/normalize.js", + "store/sqlite/lucene.js", + ].map((file) => [ + file, + createHash("sha256") + .update(readFileSync(new URL(`../../dist/${file}`, import.meta.url))) + .digest("hex"), + ]), + ), + runnerSha256: createHash("sha256") + .update(readFileSync(new URL("./combined.ts", import.meta.url))) + .digest("hex"), + configurations: results, + devPassed: eligible.length > 0, + chosen, + devRankings: chosenRows, + heldOut: eligible.length + ? report( + await run( + chosen.options, + corpus.queries.filter((q) => q.split === "test"), + ), + ) + : null, + }, + null, + 2, + ), + ) +} finally { + await store.close() +} diff --git a/docs/dev/combined-search.md b/docs/dev/combined-search.md index e4484a53d..b84860b04 100644 --- a/docs/dev/combined-search.md +++ b/docs/dev/combined-search.md @@ -1,8 +1,8 @@ # One message search: proposal for approval -Status: benchmark and plan only. No production search behavior has changed. Owner approval is -required before implementation. The proposed first release combines lexical matchers under the -Lucene grammar; meaning search follows only after a separate message-level evaluation. +Status: the owner approved the lexical experiment. Its dev evaluation failed the approved +quality gates, so CLI/MCP and SDK defaults remain unchanged. The internal matcher and measured +results are available for review; the revised semantic evaluation below needs owner approval. ## Evidence and acceptance targets @@ -42,6 +42,53 @@ back the measured limitation and a revised plan for approval. Do not tune on hel weaken labels. Freeze the dev-selected configuration before evaluating held-out; a second development cycle needs a fresh held-out set. Baseline exposure means this holdout is not blinded. +## Measured lexical experiment and revised proposal + +The internal [matcher](../../src/services/messages-combined.ts) evaluates word/stem, prefix and +corrected candidates independently, retains resolved hard predicates, fuses by rank position and +can rerank using term strength/coverage, minimum token span and phrase adjacency. It bounds depth, +AST nodes, hydrated body bytes, reranking work and elapsed time; all candidate paths check account +scope and cancellation. It is not wired to public services, CLI, MCP, bots or saved replay. + +[The dev sweep](../../bench/message-search-quality/combined-dev.json) varies all 180 combinations: +depth 50/100/300/500, RRF k 20/60/100, no reranker/coverage/proximity/phrase/all, and chat caps 0/3/5. +Every combination produces the same dev relevance metrics: + +| Mode | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | +|---|---|---|---|---| +| Strict baseline | 0.313 | 0.325 | 0.287 | 0 | +| Legacy baseline | 0.388 | 0.425 | 0.354 | 80 | +| Combined lexical | 0.363 | 0.375 | 0.326 | 0 | +| Approved minimum | 0.450 | 0.500 | 0.400 | 0 | + +Typos and beginnings reach recall 0.250 on dev, improving on strict's zero but merely matching +legacy. Sender/date controls retain full recall. Explicit AND/OR and phrases preserve strict +results and order. The lost legacy paraphrase hits came from broad any-word retrieval; removing +that noise also removes occasional useful hits. No setting clears the gates or beats legacy's +answerable metrics. Candidate depth and chat diversification do not distinguish useful messages +from newer templates with the same words and adjacency. This is a result for these tested signals +and corpus, not proof that every lexical ranking method is inadequate. + +No held-out evaluation was run for combined search because no dev configuration qualified. +No 100k-message latency claim is made; release-scale performance work follows a qualified design. +The old baseline remains committed unchanged. The smallest experimental configuration is reported +for inspection, not selected as a release recommendation; the internal defaults are provisional. + +Proposed next phase, requiring approval: measure a local semantic reranker over the bounded, +filter-valid lexical candidates before changing the public default. First define its model, +installation/readiness behavior and cost budget, without an automatic download or network call. +Evaluate per-message text against the whole query; compare it with chunk-to-message semantic +retrieval only if lexical candidate recall itself is inadequate. Do not reuse the conversation +0.80 cosine floor as a message threshold. + +Keep these relevance labels and release gates. Add separately authored question-form dev queries +that explicitly ask for the decision/policy when that is the intent; do not substitute them for +this keyword-query benchmark or relabel distractors. The existing `intent` field contains the +answer text and must never be passed to retrieval or a reranker. Create fresh held-out topic/query +families before tuning the new approach, freeze the dev choice, then evaluate held-out once. +If semantic ranking also fails, report that result rather than relax the release gates implicitly. +All CLI/MCP removal, SDK/saved conversion and consumer releases below remain pending quality gates. + ## Query behavior CLI and MCP accept one language, Lucene, and remove the `--language` option. Parse and resolve once @@ -177,7 +224,7 @@ This proposal includes the separately agreed breaking window required by the version, updating generated `docs/commands.md`, skills and breaking changelog entries. Avoid other consumer `docs/*.md` prose owned by `docs/reader-standards`. Verify unreleased builds with `bin/try-messaging` in consumer worktrees; no committed `file:` dependency. Arrange same-day - consumer moves when the breaking shared release lands. No release/publish is part of this plan-only task. + consumer moves when the breaking shared release lands. No release/publish proceeds while the quality gates are unmet. No migration number is needed now. If saved conversion or later semantic work requires schema changes, reserve a number first through [coordination](COORDINATION.md#store-migrations). diff --git a/src/search/lucene/parser.ts b/src/search/lucene/parser.ts index b89473885..40741e1b0 100644 --- a/src/search/lucene/parser.ts +++ b/src/search/lucene/parser.ts @@ -262,3 +262,18 @@ export const parseLucene = ( expect("EOF") return { version: 1, language: "lucene-v1", root } } + +/** Bare terms eligible for forgiving retrieval; explicit syntax keeps strict semantics. */ +export const implicitTextTerms = (text: string): Set => { + const tokens = tokensOf(text) + if (tokens.some(({ kind }) => ["AND", "OR", "NOT", "+", "-", "!", "(", ")"].includes(kind))) return new Set() + return new Set( + tokens.flatMap((token, index) => + token.kind === "term" && + ![":", "=", "<", ">", "<=", ">="].includes(tokens[index - 1]?.kind ?? "") && + ![":", "=", "<", ">", "<=", ">="].includes(tokens[index + 1]?.kind ?? "") + ? [token.span.start] + : [], + ), + ) +} diff --git a/src/services/messages-combined.test.ts b/src/services/messages-combined.test.ts new file mode 100644 index 000000000..3f932a5ff --- /dev/null +++ b/src/services/messages-combined.test.ts @@ -0,0 +1,341 @@ +import { mkdtempSync } from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { afterEach, describe, expect, it, vi } from "vitest" +import type { Message } from "../domain/models.js" +import { implicitTextTerms, parseLucene } from "../search/lucene/parser.js" +import { QUERY_LIMITS } from "../search/lucene/types.js" +import { type AccountKey, type MessageStore, openStore } from "../store/store.js" +import { searchStore } from "./messages.js" +import { COMBINED_DEFAULTS, searchCombined } from "./messages-combined.js" + +const account: AccountKey = { provider: "synthetic", account: "500" } +const stores: MessageStore[] = [] +afterEach(async () => { + for (const store of stores.splice(0)) await store.close() + vi.restoreAllMocks() +}) +const message = (id: string, text: string, extra: Partial = {}): Message => ({ + id, + chatId: "101", + senderId: "700", + senderName: "Mira", + timestamp: `2026-10-02T12:00:${id.padStart(2, "0")}.000Z`, + editedAt: null, + text, + outgoing: false, + attachments: [], + replyTo: null, + forwardedFrom: null, + reactions: null, + ...extra, +}) +const opened = async ( + messages = [message("1", "Aurora rollout decision"), message("2", "Aurora rolling decision")], +) => { + const store = await openStore({ path: join(mkdtempSync(join(tmpdir(), "combined-test-")), "quality.db") }) + stores.push(store) + await store.saveChats(account, [ + { id: "101", title: "Decisions", kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 20 }, + { id: "102", title: "Other", kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 20 }, + ]) + await store.saveMessages(account, "101", messages, { via: "synthetic" }) + await store.fillSearchIndex({}) + await store.fillStems({}) + return store +} +const request = (text: string) => ({ text, limit: 10, timezone: "UTC" }) +const ids = (found: { items: { id: string }[] }) => found.items.map((h) => h.id) + +describe("internal combined search", () => { + it("merges whole and prefix matches even when whole words already match, with positive scores and labels", async () => { + const store = await opened() + const found = await searchCombined(store, account, request("aurora roll")) + expect(ids(found).sort()).toEqual(["1", "2"]) + expect(found.items).toEqual( + expect.arrayContaining([expect.objectContaining({ match: "beginnings", matches: ["beginnings"] })]), + ) + for (const hit of found.items) expect(hit.score).toBeGreaterThan(0) + expect(found.query.combined).toMatchObject({ expanded: true, truncated: false }) + expect(found.coverage?.accounts).toEqual([account]) + const mixed = await searchCombined(store, account, request("aurora rollout")) + expect(mixed.items[0]).toMatchObject({ match: "words", matches: ["words"] }) + expect(new Set(mixed.items.map((h) => h.locator)).size).toBe(mixed.items.length) + }) + + it("reports only a used correction and never falls back to any-word or substring", async () => { + const store = await opened() + const fixed = await searchCombined(store, account, request("aurora rolluot")) + expect(ids(fixed)).toEqual(["1"]) + expect(fixed.items[0]).toMatchObject({ match: "corrected", matches: ["corrected"] }) + expect(fixed.corrections).toEqual([{ from: "rolluot", to: ["rollout"] }]) + for (const text of ["aurora quantum unicorn", "lout", "aurora rolluot purple"]) { + const found = await searchCombined(store, account, request(text)) + expect(found.items).toEqual([]) + expect(found.corrections).toEqual([]) + } + }) + + it.each([ + "aurora AND rolluot", + "aurora OR rolluot", + "aurora NOT rollout", + "+aurora rolluot", + "aurora -rollout", + "aurora !rollout", + '"aurora rolluot"', + "text:rolluot", + "exact:rolluot", + "(aurora rolluot)", + ])("preserves the strict execution for %s", async (text) => { + const store = await opened() + const strict = await searchStore(store, account, { ...request(text), language: "lucene" }) + const combined = await searchCombined(store, account, request(text)) + expect(combined.items).toEqual(strict.items) + expect(combined.corrections).toEqual([]) + }) + + it("preserves phrases, sender, chat, date, source and only-message boundaries across corrections", async () => { + const store = await opened([ + message("1", "Aurora rollout green light"), + message("2", "Aurora rollout light green"), + message("3", "Aurora rollout green light", { senderId: "701" }), + message("4", "Aurora rollout green light", { timestamp: "2026-10-03T12:00:00.000Z" }), + ]) + await store.saveMessages(account, "102", [message("5", "Aurora rollout green light", { chatId: "102" })], { + via: "synthetic", + }) + const other = { provider: "synthetic", account: "600" } + await store.saveChats(other, [ + { id: "101", title: "Other account", kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 10 }, + ]) + await store.saveMessages(other, "101", [message("6", "Aurora rollout green light")], { via: "synthetic" }) + const found = await searchCombined( + store, + account, + request('aurora rolluot "green light" from:700 chat:101 date:2026-10-02'), + ) + expect(ids(found)).toEqual(["1"]) + expect( + ids(await searchCombined(store, account, { ...request("aurora rolluot"), only: [{ chatId: "101", id: "3" }] })), + ).toEqual(["3"]) + expect( + ids( + await searchCombined(store, account, { + ...request("aurora rolluot"), + senders: [{ provider: "synthetic", id: "701" }], + }), + ), + ).toEqual(["3"]) + const all = await searchCombined(store, account, { ...request("aurora rolluot"), source: "all" }) + expect(all.items.some((h) => h.id === "6")).toBe(true) + expect((await searchCombined(store, account, request("aurora rolluot"))).items.some((h) => h.id === "6")).toBe( + false, + ) + }) + + it("retains strict exact and AST requests and the legacy SDK default", async () => { + const store = await opened() + const ast = parseLucene("aurora rollout") + const exact = { ...request("aurora roll"), exact: true } + expect((await searchCombined(store, account, exact)).items).toEqual( + (await searchStore(store, account, { ...exact, language: "lucene" })).items, + ) + const structured = { ast, limit: 10, timezone: "UTC" } + expect((await searchCombined(store, account, structured)).items).toEqual( + (await searchStore(store, account, structured)).items, + ) + expect( + (await searchStore(store, account, { text: "aurora quantum unicorn", limit: 10 })).items.length, + ).toBeGreaterThan(0) + expect((await searchCombined(store, account, request("aurora quantum unicorn"))).items).toEqual([]) + }) + + it("matches stems and corrections independently, including Cyrillic", async () => { + const store = await opened([ + message("1", "Север сертификат обновление"), + message("2", "Aurora running safely"), + message("3", "Aurora runners safely"), + ]) + expect(ids(await searchCombined(store, account, request("север сертифкат")))).toEqual(["1"]) + const stemmed = await searchCombined(store, account, request("aurora run")) + expect(ids(stemmed).sort()).toEqual(["2", "3"]) + expect(stemmed.query.stemming).toMatchObject({ applied: true }) + }) + + it.each([1, 10, 50, 1000])( + "honors limit %i, exposes truncation, hydrates context and sorts newest chronologically", + async (limit) => { + const store = await opened([ + message("1", "Aurora rollout decision"), + message("2", "Aurora rollout"), + message("3", "Aurora rollouts"), + ]) + const found = await searchCombined( + store, + account, + { ...request("aurora roll"), limit, newest: true, context: 1 }, + {}, + { ...COMBINED_DEFAULTS, candidateDepth: 1, chatCap: 3 }, + ) + expect(found.items.length).toBeLessThanOrEqual(limit) + expect(found.items.every((h) => h.context)).toBe(true) + expect(found.query.order).toBe("newest") + const timestamps = found.items.map((h) => h.timestamp) + expect(timestamps).toEqual([...timestamps].sort().reverse()) + if (limit === 1) { + expect(found.hasMore).toBe(true) + expect(found.query.combined.truncated).toBe(true) + } + }, + ) + + it("diversifies chats without dropping deferred matches; one-chat requests keep their order", async () => { + const store = await opened([ + message("1", "Aurora rollout"), + message("2", "Aurora rollout"), + message("3", "Aurora rollout"), + message("4", "Aurora rollout"), + ]) + await store.saveMessages( + account, + "102", + [message("5", "Aurora rollout context long enough to rank behind", { chatId: "102" })], + { via: "synthetic" }, + ) + const options = { ...COMBINED_DEFAULTS, chatCap: 3 } + const diverse = await searchCombined(store, account, { ...request("aurora rollout"), limit: 5 }, {}, options) + expect(diverse.items.slice(0, 3).some((h) => h.chatId === "102")).toBe(true) + expect(diverse.items.length).toBe(5) + const scoped = { ...request("aurora rollout"), chat: "101" } + expect(ids(await searchCombined(store, account, scoped, {}, options))).toEqual( + ids(await searchCombined(store, account, scoped)), + ) + }) + + it("handles filters-only, an empty authorized scope, and unsupported internal requests", async () => { + const store = await opened() + expect(ids(await searchCombined(store, account, request("chat:101")))).toEqual(["2", "1"]) + expect((await searchCombined(store, account, { ...request("aurora"), source: "bots" })).items).toEqual([]) + for (const extras of [ + { language: "legacy" as const }, + { pattern: /aurora/iu }, + { thread: { before: 1, after: 1 } }, + ]) + await expect(searchCombined(store, account, { ...request("aurora"), ...extras })).rejects.toMatchObject({ + code: "validation_error", + }) + await expect(searchCombined({ ...store, matchQuery: undefined }, account, request("aurora"))).rejects.toThrow( + "does not support Lucene", + ) + }) + + it.each([ + { candidateDepth: 0 }, + { candidateDepth: 1001 }, + { rrfK: 0 }, + { rrfK: NaN }, + { chatCap: -1 }, + { chatCap: 0.5 }, + ])("refuses invalid options %j", async (options) => { + const store = await opened() + await expect( + searchCombined(store, account, request("aurora"), {}, { ...COMBINED_DEFAULTS, ...options }), + ).rejects.toMatchObject({ code: "validation_error" }) + }) + + it("honors cancellation before retrieval and between sources", async () => { + const store = await opened() + const before = new AbortController() + before.abort() + await expect(searchCombined(store, account, { ...request("aurora"), signal: before.signal })).rejects.toMatchObject( + { details: { reason: "query_aborted" } }, + ) + const during = new AbortController() + const match = store.matchQuery + if (!match) throw new Error("test store must support Lucene") + const interrupted = { + ...store, + matchQuery: vi.fn(async (...args: Parameters) => { + const found = await match(...args) + during.abort() + return found + }), + } + await expect( + searchCombined(interrupted, account, { ...request("aurora"), signal: during.signal }), + ).rejects.toMatchObject({ details: { reason: "query_aborted" } }) + expect(interrupted.matchQuery).toHaveBeenCalledTimes(1) + }) +}) + +it("keeps explicit wildcard and regex predicates as hard constraints beside expanded words", async () => { + const store = await opened([ + message("1", "Aurora rollout decision"), + message("2", "Aurorax rolling decision"), + message("3", "Aurora unrelated decision"), + ]) + expect(ids(await searchCombined(store, account, request("auror roll*"))).sort()).toEqual(["1", "2"]) + expect(ids(await searchCombined(store, account, request("auror text:/rollout/")))).toEqual(["1"]) +}) + +it.each(["none", "coverage", "proximity", "phrase", "all"] as const)( + "scores the %s ablation without leaking fusion state", + async (rerank) => { + const store = await opened([ + message("1", "Aurora widely separated rollout"), + message("2", "Aurora rollout decision"), + ]) + const found = await searchCombined(store, account, request("aurora rollout"), {}, { ...COMBINED_DEFAULTS, rerank }) + expect(found.items).toHaveLength(2) + expect(found.items.every((hit) => Number.isFinite(hit.score) && Number(hit.score) > 0)).toBe(true) + expect(found.items[0]).not.toHaveProperty("fused") + if (rerank === "phrase" || rerank === "all" || rerank === "proximity") expect(ids(found)[0]).toBe("2") + }, +) + +it("bounds hydrated candidate bytes, reranking work and elapsed time", async () => { + const store = await opened() + const original = await searchStore(store, account, { ...request("aurora"), language: "lucene" }) + const first = original.items[0] + if (!first) throw new Error("fixture needs a hit") + const oversized = { + ...store, + matchQuery: async () => ({ + items: [{ ...first, score: 1, text: "a".repeat(QUERY_LIMITS.bodyBytes + 1) }], + hasMore: false, + }), + } + await expect(searchCombined(oversized, account, request("aurora"))).rejects.toMatchObject({ + details: { reason: "query_limit", budget: "body bytes" }, + }) + const expensive = { + ...store, + matchQuery: async () => ({ items: [{ ...first, score: 1, text: "aurora ".repeat(100000) }], hasMore: false }), + } + await expect( + searchCombined(expensive, account, request(Array.from({ length: 110 }, () => "aurora").join(" "))), + ).rejects.toMatchObject({ details: { reason: "query_limit", budget: "work" } }) + vi.spyOn(performance, "now") + .mockReturnValueOnce(0) + .mockReturnValue(QUERY_LIMITS.milliseconds + 1) + await expect(searchCombined(store, account, request("aurora"))).rejects.toMatchObject({ + details: { reason: "query_limit", budget: "milliseconds" }, + }) +}) + +it("refuses corrected AST expansion past the parser node budget", async () => { + const store = await opened() + await expect( + searchCombined(store, account, request(Array.from({ length: 130 }, () => "rolluot").join(" "))), + ).rejects.toMatchObject({ details: { reason: "query_limit", budget: "nodes" } }) +}) + +describe("implicit term intent", () => { + it("does not mistake filter values, quotes, escaped operator words or ranges for explicit operators", () => { + expect([...implicitTextTerms('aurora chat:101 "AND OR" date:[2026-10-01 TO *] roll')]).toEqual([0, 48]) + expect([...implicitTextTerms("aurora from:AND roll")]).toEqual([]) + expect([...implicitTextTerms('aurora from:"AND" roll')]).toEqual([0, 18]) + expect([...implicitTextTerms("aurora text:roll")]).toEqual([0]) + }) +}) diff --git a/src/services/messages-combined.ts b/src/services/messages-combined.ts new file mode 100644 index 000000000..cef7eb0f2 --- /dev/null +++ b/src/services/messages-combined.ts @@ -0,0 +1,314 @@ +import { CliError } from "@leemour/cli-core" +import type { Messenger } from "../cli/messenger/context.js" +import { parseLocator } from "../domain/locator.js" +import type { Page } from "../domain/models.js" +import { correctWords } from "../search/correct.js" +import { implicitTextTerms } from "../search/lucene/parser.js" +import { PRESET_VERSION } from "../search/lucene/presets.js" +import { FIELD_VERSION } from "../search/lucene/registry.js" +import type { QueryExecution, ResolvedNode } from "../search/lucene/resolved.js" +import { exhausted, QUERY_LIMITS, walkQuery } from "../search/lucene/types.js" +import { normalize } from "../store/normalize.js" +import type { AccountKey, MessageStore, ScoredHit } from "../store/store.js" +import type { FoundMessage, SearchFound, SearchQuery } from "./messages.js" +import type { QueryMetadata } from "./messages-search.js" +import { coverageOf, prepareLucene } from "./messages-search.js" + +type Source = "words" | "beginnings" | "corrected" +export interface CombinedOptions { + candidateDepth: number + rrfK: number + rerank: "none" | "coverage" | "proximity" | "phrase" | "all" + chatCap: number +} +export const COMBINED_DEFAULTS: CombinedOptions = { candidateDepth: 300, rrfK: 60, rerank: "all", chatCap: 0 } +export type CombinedFound = SearchFound & { + query: QueryMetadata & { combined: { candidateDepth: number; truncated: boolean; expanded: boolean } } +} +type Ranked = ScoredHit & { match: Source; matches: Source[]; fused: number; score: number } +const tokensOf = (text: string) => normalize(text).match(/[\p{L}\p{N}\p{Co}\p{M}]+/gu) ?? [] + +/** Internal measured entry point; public search stays on its existing compatibility paths. */ +export const searchCombined = async ( + store: MessageStore, + account: AccountKey, + request: SearchQuery, + messenger: Partial> = {}, + options: CombinedOptions = COMBINED_DEFAULTS, +): Promise => { + if (request.pattern || request.language === "legacy") + throw new CliError("validation_error", "combined search takes Lucene text or AST, not legacy or JavaScript regex") + if (request.thread) throw new CliError("validation_error", "internal combined evaluation does not hydrate threads") + if ( + !Number.isInteger(options.candidateDepth) || + options.candidateDepth < 1 || + options.candidateDepth > 1000 || + !Number.isFinite(options.rrfK) || + options.rrfK <= 0 || + !Number.isInteger(options.chatCap) || + options.chatCap < 0 || + !["none", "coverage", "proximity", "phrase", "all"].includes(options.rerank) + ) + throw new CliError("validation_error", "invalid internal combined search bounds") + const prepared = await prepareLucene(store, account, request, messenger) + const execute = store.matchQuery + if (!execute) throw new CliError("validation_error", "this store does not support Lucene search") + const execution = prepared.execution + const started = performance.now() + const check = () => { + if (request.signal?.aborted) + throw new CliError("validation_error", "search was aborted", { reason: "query_aborted", complete: false }) + if (performance.now() - started > QUERY_LIMITS.milliseconds) exhausted("milliseconds") + } + check() + const spans = + request.ast === undefined && !request.exact && request.text !== undefined + ? implicitTextTerms(request.text) + : new Set() + const eligible = walkQuery(execution.root).filter( + (leaf) => + spans.has(leaf.span.start) && + leaf.field === "text" && + leaf.operator === "term" && + tokensOf(leaf.value).length === 1 && + /^[\p{L}\p{N}]+$/u.test(normalize(leaf.value)), + ) + const expanding = eligible.length > 0 + const depth = Math.max(request.limit, options.candidateDepth) + const queryTerms = walkQuery(execution.root) + .filter( + (leaf) => + (leaf.field === "text" || leaf.field === "exact") && (leaf.operator === "term" || leaf.operator === "phrase"), + ) + .flatMap((leaf) => tokensOf(leaf.value)) + const corrections = expanding + ? await correctWords(store, [...new Set(eligible.map((leaf) => normalize(leaf.value)))]) + : new Map() + check() + const bySpan = new Map(eligible.map((leaf) => [leaf.span.start, leaf])) + const transform = (node: ResolvedNode, source: Source, exact = false): ResolvedNode => { + if (node.kind === "boolean") + return { + ...node, + clauses: node.clauses.map((clause) => ({ ...clause, node: transform(clause.node, source, exact) })), + } + if (exact && node.field === "text" && (node.operator === "term" || node.operator === "phrase")) + return { ...node, field: "exact" } + if (!bySpan.has(node.span.start)) return node + if (source === "beginnings" && [...normalize(node.value)].length >= 3) + return { ...node, operator: "wildcard", value: `${normalize(node.value)}*` } + if (source !== "corrected") return node + const fixes = corrections.get(normalize(node.value)) + if (!fixes) return node + const clauses = fixes.map((value) => ({ occur: "should" as const, node: { ...node, field: "exact", value } })) + return { kind: "boolean", clauses, span: node.span } + } + const read = async (root: ResolvedNode, limit: number): Promise> => { + check() + const nodes = (node: ResolvedNode): number => + node.kind === "predicate" ? 1 : 1 + node.clauses.reduce((sum, clause) => sum + nodes(clause.node), 0) + if (nodes(root) > QUERY_LIMITS.nodes) exhausted("nodes") + const found = execution.accounts.length + ? await execute({ ...execution, root, limit }) + : { items: [], hasMore: false } + check() + return found + } + // Explicit syntax and AST retain the exact production execution and order. + if (!expanding) { + const found = await read(execution.root, request.limit) + return finish(found.items, found.hasMore, [], false, false) + } + const exact = await read(transform(execution.root, "words", true), depth) + const stemmed = await read(execution.root, depth) + const prefix = await read(transform(execution.root, "beginnings"), depth) + const fixed = corrections.size + ? await read(transform(execution.root, "corrected"), depth) + : { items: [], hasMore: false } + const fixedPrefixRoot = transform(execution.root, "corrected") + const correctedBeginnings = (node: ResolvedNode): ResolvedNode => + node.kind === "boolean" + ? { ...node, clauses: node.clauses.map((clause) => ({ ...clause, node: correctedBeginnings(clause.node) })) } + : bySpan.has(node.span.start) && [...normalize(node.value)].length >= 3 + ? { ...node, field: "exact", operator: "wildcard", value: `${normalize(node.value)}*` } + : node + const fixedPrefix = corrections.size + ? await read(correctedBeginnings(fixedPrefixRoot), depth) + : { items: [], hasMore: false } + const ranked = new Map() + const add = (lists: Page[], source: Source) => { + const shares = new Map() + for (const list of lists) + list.items.forEach((hit, rank) => { + const share = 1 / (options.rrfK + rank + 1) + const old = shares.get(hit.locator) + if (!old || share > old.share) shares.set(hit.locator, { hit, share }) + }) + for (const { hit, share } of shares.values()) { + const old = ranked.get(hit.locator) + if (old && source === "beginnings") continue + if (old) { + old.fused += share + old.matches.push(source) + } else ranked.set(hit.locator, { ...hit, match: source, matches: [source], fused: share, score: 0 }) + } + } + add([exact, stemmed], "words") + add([prefix], "beginnings") + add([fixed, fixedPrefix], "corrected") + let bytes = 0 + const work = { used: 0 } + for (const hit of ranked.values()) { + bytes += new TextEncoder().encode(hit.text).byteLength + if (bytes > QUERY_LIMITS.bodyBytes) exhausted("body bytes") + check() + hit.score = hit.fused * (options.rrfK + 1) + signals(hit, queryTerms, corrections, execution, options.rerank, work) + check() + } + const sorted = [...ranked.values()].sort((a, b) => + request.newest + ? b.timestamp.localeCompare(a.timestamp) || a.locator.localeCompare(b.locator) + : b.score - a.score || + b.fused - a.fused || + b.timestamp.localeCompare(a.timestamp) || + a.locator.localeCompare(b.locator), + ) + const ordered: Ranked[] = [] + const deferred: Ranked[] = [] + const counts = new Map() + const cap = + execution.chat || request.newest || !options.chatCap ? 0 : Math.ceil((options.chatCap * request.limit) / 10) + for (const hit of sorted) { + const { provider, account } = parseLocator(hit.locator) + const chat = JSON.stringify([provider, account, hit.chatId]) + const count = counts.get(chat) ?? 0 + if (cap && count >= cap) deferred.push(hit) + else { + ordered.push(hit) + counts.set(chat, count + 1) + } + } + ordered.push(...deferred) + const truncated = [exact, stemmed, prefix, fixed, fixedPrefix].some((page) => page.hasMore) + const listed = ordered.slice(0, request.limit) + const usedCorrections = listed.some((hit) => hit.matches.includes("corrected")) + ? [...corrections] + .filter(([, fixes]) => + listed.some( + (hit) => + hit.matches.includes("corrected") && + tokensOf(hit.text).some((token) => fixes.some((fix) => token.startsWith(fix))), + ), + ) + .map(([from, to]) => ({ from, to })) + : [] + return finish( + listed.map(({ fused: _fused, ...hit }) => hit), + ordered.length > request.limit || truncated, + usedCorrections, + truncated, + true, + ) + + async function finish( + items: FoundMessage[], + hasMore: boolean, + listed: { from: string; to: string[] }[], + truncated: boolean, + expanded: boolean, + ): Promise { + const { completeness, coverage } = await coverageOf(store, prepared, messenger) + const hydrated = await Promise.all( + items.map(async (hit) => { + if (!request.context) return hit + const { provider, account } = parseLocator(hit.locator) + return { + ...hit, + context: await store.around({ provider, account }, hit.chatId, hit.id, { + before: request.context, + after: request.context, + }), + } + }), + ) + return { + items: hydrated, + hasMore, + corrections: listed, + completeness, + coverage, + wordsReady: prepared.wordsReady, + stemsReady: prepared.stemsReady, + query: { + language: "lucene-v1", + version: 1, + fieldsVersion: FIELD_VERSION, + presetVersion: PRESET_VERSION, + timezone: prepared.timezone, + order: request.newest ? "newest" : "relevance", + ...(prepared.stemming ? { stemming: prepared.stemming } : {}), + combined: { candidateDepth: depth, truncated, expanded }, + }, + } + } +} + +const signals = ( + hit: ScoredHit, + terms: string[], + corrections: Map, + execution: QueryExecution, + mode: CombinedOptions["rerank"], + work: { used: number }, +): number => { + if (mode === "none" || !terms.length) return 0 + const tokens = tokensOf(hit.text) + work.used += terms.length * tokens.length + if (work.used > QUERY_LIMITS.work) exhausted("work") + const positions = terms.map((term) => + tokens.flatMap((token, index) => { + const strength = + token === term + ? 1 + : execution.stemmer?.stemToken(token) === execution.stemmer?.stemToken(term) && execution.stemmer + ? 0.9 + : [...term].length >= 3 && token.startsWith(term) + ? 0.8 + : corrections.get(term)?.some((fix) => token.startsWith(fix)) + ? 0.7 + : 0 + return strength ? [{ index, strength }] : [] + }), + ) + const coverage = + positions.reduce( + (sum, matches) => sum + matches.reduce((maximum, match) => Math.max(maximum, match.strength), 0), + 0, + ) / terms.length + const events = positions + .flatMap((matches, term) => matches.map(({ index }) => ({ index, term }))) + .sort((a, b) => a.index - b.index) + const held = new Map() + let left = 0, + span = Infinity + for (const [right, event] of events.entries()) { + held.set(event.term, (held.get(event.term) ?? 0) + 1) + while (held.size === terms.length && left <= right) { + const first = events[left] as { index: number; term: number } + span = Math.min(span, event.index - first.index + 1) + const count = (held.get(first.term) ?? 1) - 1 + if (count) held.set(first.term, count) + else held.delete(first.term) + left++ + } + } + const proximity = Number.isFinite(span) ? Math.min(1, terms.length / span) : 0 + const phrase = tokens.some((_, index) => terms.every((term, offset) => tokens[index + offset] === term)) ? 1 : 0 + return mode === "coverage" + ? coverage * 4 + : mode === "proximity" + ? proximity + : mode === "phrase" + ? phrase + : coverage * 4 + proximity + phrase +} From 10cdb4797ccdc384b9cd178b45eef90974f44ed4 Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 15:25:20 +0200 Subject: [PATCH 04/14] test(search): measure candidate recall and local joint reranking --- bench/message-search-quality/README.md | 93 + .../augmented-baseline.json | 11550 ++++++ .../crossencoder-dev.json | 30187 ++++++++++++++++ bench/message-search-quality/crossencoder.ts | 66 + .../generate-questions.py | 50 + .../prepare-reranker.py | 25 + bench/message-search-quality/questions.json | 4544 +++ bench/message-search-quality/reranker.json | 28 + bench/message-search-quality/run.ts | 7 + .../message-search-quality/semantic-dev.json | 27423 ++++++++++++++ bench/message-search-quality/semantic.ts | 422 + bench/message-search-quality/tsconfig.json | 12 +- docs/dev/combined-search.md | 59 +- 13 files changed, 74456 insertions(+), 10 deletions(-) create mode 100644 bench/message-search-quality/augmented-baseline.json create mode 100644 bench/message-search-quality/crossencoder-dev.json create mode 100644 bench/message-search-quality/crossencoder.ts create mode 100644 bench/message-search-quality/generate-questions.py create mode 100644 bench/message-search-quality/prepare-reranker.py create mode 100644 bench/message-search-quality/questions.json create mode 100644 bench/message-search-quality/reranker.json create mode 100644 bench/message-search-quality/semantic-dev.json create mode 100644 bench/message-search-quality/semantic.ts diff --git a/bench/message-search-quality/README.md b/bench/message-search-quality/README.md index 206be31ac..136da54df 100644 --- a/bench/message-search-quality/README.md +++ b/bench/message-search-quality/README.md @@ -98,3 +98,96 @@ combined held-out result is produced when dev fails. The original baseline is pr The [revised proposal](../../docs/dev/combined-search.md#measured-lexical-experiment-and-revised-proposal) requests a semantic evaluation before public adoption. Timings are from this small corpus on a shared machine; they are not release-scale latency evidence. + +## Candidate recall and semantic ranking + +[semantic.ts](semantic.ts) measures the lexical candidate pool before reranking. The original +corpus and labels remain unchanged. [questions.json](questions.json) adds eight answerable and four +no-answer questions on the existing dev topics, plus four fresh held-out topic families with 608 +new synthetic messages and twelve questions. [generate-questions.py](generate-questions.py) +reproduces this supplement. Its input hashes are frozen before model inference and parameter selection. +Query `intent` and relevance labels are never supplied to either model. + +Dev candidate recall is 0.4375 at depth 50, 0.7500 at 100, and 0.9000 at 300/500. Depth 300 is the +smallest tested pool reaching maximum recall. Every non-paraphrase category has full candidate +recall at that depth; paraphrases have zero. The pool size measures returned candidates from the +internal service, rather than the union of every raw matcher list. + +To measure the already installed, pinned e5-small model: + +```sh +pnpm build +pnpm exec tsc -p bench/message-search-quality/tsconfig.json +node bench/message-search-quality/semantic.ts > bench/message-search-quality/semantic-dev.json +``` + +The benchmark verifies the local model's hashes before opening it. e5-small cosine reranking +reduces original dev keyword recall@10 from 0.3625 to 0.2750, MRR from 0.3750 to 0.3000, and nDCG +from 0.3264 to 0.2674. On the additional answerable questions it improves ranking when supplied +manual topic anchors for retrieval, but every tested threshold preserving zero no-answer hits +reduces answerable recall to zero. No e5 held-out evaluation is performed because dev fails. + +The second experiment uses the official +[mmarco-mMiniLMv2-L12-H384-v1 model](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1) +to jointly score query-message pairs. [reranker.json](reranker.json) pins an Apache-2.0 ONNX export, +its revision, tokenizer/config files and SHA-256 hashes. Its identity activation is retained: +scores are raw logits, not cosine or calibrated probabilities. The benchmark-only adapter is +[crossencoder.ts](crossencoder.ts); it uses the installed tokenizer and WebAssembly ONNX runtime, +checks input/output shapes and rejects overlength pairs instead of silently truncating them. + +Model preparation is an explicit, separate network step, totaling about 136 MB. Use a directory +outside the real store and repository, then run the offline evaluation: + +```sh +python bench/message-search-quality/prepare-reranker.py /tmp/message-search-reranker-1427fd6 +MESSAGE_RERANKER_DIR=/tmp/message-search-reranker-1427fd6 \ + node bench/message-search-quality/semantic.ts --crossencoder > bench/message-search-quality/crossencoder-dev.json +``` + +The evaluation cannot download or install a model. Preparation and benchmark stores retain their +files for inspection; cleanup is a separate task. Both models score only bounded lexical candidates. +Explicit Boolean/phrase/AST requests retain the existing strict ranking. All parameters are selected +on dev; held-out inference happens only after the dev ranking/no-answer gates pass. + +The question-form experiment deliberately separates two measurements: direct full-question Lucene +retrieval, which currently finds no candidates, and reranking a manually supplied topic shortlist. +Manual topic anchors are diagnostic inputs, not an implemented natural-language query parser. +These results do not establish end-to-end question search. Questions asking for an unknown auditor +provide harder no-answer controls than the original requests with absent topic words. + +### Measured query-message ranking + +[crossencoder-dev.json](crossencoder-dev.json) records the frozen model, depth 300, raw score floor +2 selected for the question diagnostic, exact hashes, per-query rankings, and both held-out reports. +Keyword requests use reranking without a score floor; their no-answer candidate pools are empty. +The question floor is fitted on dev only and is not a general answerability or confidence guarantee. + +| Original query split | Mode | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | +|---|---|---|---|---|---| +| dev | Strict | 0.313 | 0.325 | 0.287 | 0 | +| dev | Legacy | 0.388 | 0.425 | 0.354 | 80 | +| dev | Combined + joint reranker | 0.525 | 0.553 | 0.477 | 0 | +| held-out | Strict | 0.263 | 0.325 | 0.219 | 0 | +| held-out | Legacy | 0.338 | 0.450 | 0.299 | 80 | +| held-out | Combined + joint reranker | 0.438 | 0.488 | 0.405 | 0 | + +The original keyword held-out result beats both previous modes but misses the approved recall +0.45 and MRR 0.50 gates. nDCG and no-answer gates pass. No parameters are changed after reading +these results. The fresh question held-out diagnostic has recall 0.6875, MRR 1.0, nDCG 0.8660 and +zero false hits on four no-answer queries, using manual anchors. This small synthetic set does not +establish general answerability or a functioning full-question retrieval path. + +The semantic experiment indexes both the original corpus and fresh supplemental messages. To +control for the changed archive, strict/legacy results on that same 3,224-message archive are in +[augmented-baseline.json](augmented-baseline.json); their original-query metrics equal the old +baseline. Reproduce that comparison without changing the original baseline file: + +```sh +node bench/message-search-quality/run.ts --include-questions > bench/message-search-quality/augmented-baseline.json +``` + +The joint model scores about 10–12 ms per pair in these runs on eight WebAssembly threads, plus +roughly a second to load. Scoring 300 uncached pairs therefore costs about 3–4 seconds, exceeding +the proposed 250 ms latency gate. This is measured benchmark inference, not production p95 or +an optimized batch/native implementation. Exact-pair caching helps repeated identical requests; +it does not remove the cost of a new query. No public service/default or consumer version changed. diff --git a/bench/message-search-quality/augmented-baseline.json b/bench/message-search-quality/augmented-baseline.json new file mode 100644 index 000000000..cae2e41f6 --- /dev/null +++ b/bench/message-search-quality/augmented-baseline.json @@ -0,0 +1,11550 @@ +{ + "schemaVersion": 1, + "k": 10, + "node": "v24.19.0", + "timezone": "UTC", + "storePath": "/tmp/message-search-quality-MkWCIS/quality.db", + "archiveSupplementSha256": "390ff1475964d982a06afaf9e2f6f840f5b8304fcffe58ebedd5a0ea8d54478e", + "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", + "runnerSha256": "4b6fe3eccec01b1f95704b4447aa7647070cbc213fd524f6f665b630ca7ef507", + "productionSha256": { + "services/messages.js": "62f927cddd912275bb46a7c990d64881c2b4a22eac12d54f484fd5ca4223a3e4", + "services/messages-search.js": "35eb4ed1dcc002a80985327ffa9f6bd25a5ee6b1a31f3024203a38b6605c7d2a", + "search/search.js": "e1c7975b1573936515244fdc60454a699552afde89363b517d35974e115384b6", + "search/query.js": "89a7030d749f06ca61b9867bf29db4244617943e14c383aa5a09fd40ed819772", + "search/correct.js": "54e5089f217840dd9e27e6e89c29486c2d8704850af16dafd1b8410998066270", + "search/stem.js": "84475baa966a17595c480015668dc77da60409a00c391aa9ca6a718433386007", + "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", + "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964", + "store/sqlite/words.js": "be7c86aa22f8a308ce2fba664801ffd65da4aad444f1a8e62533548980a9e5b3" + }, + "messages": 3224, + "queries": 96, + "metrics": { + "dev": { + "lucene": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3125, + "mrr": 0.325, + "ndcgAt10": 0.2865560469084554, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.25, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.9491768952476267, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + } + } + }, + "legacy": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3875, + "mrr": 0.425, + "ndcgAt10": 0.3539325811955994, + "noAnswerFalseHits": 80, + "noAnswerQueryHitRate": 1, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.25, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.9491768952476267, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.25, + "mrr": 0.5, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + } + } + } + }, + "test": { + "lucene": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.2625, + "mrr": 0.325, + "ndcgAt10": 0.21917951262131133, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.8475306857428799, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": 0 + } + } + }, + "legacy": { + "queries": 48, + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3375, + "mrr": 0.45, + "ndcgAt10": 0.2992618230965487, + "noAnswerFalseHits": 80, + "noAnswerQueryHitRate": 1, + "categories": { + "words": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "typo": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "beginnings": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "phrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "from": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "chat": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "date": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.8475306857428799, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "and": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "or": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.125, + "mrr": 0.25, + "ndcgAt10": 0.06885288809404666, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "paraphrase": { + "queries": 4, + "answerable": 4, + "noAnswerQueries": 0, + "recallAt10": 0.5, + "mrr": 0.75, + "ndcgAt10": 0.66311732856428, + "noAnswerFalseHits": 0, + "noAnswerQueryHitRate": null + }, + "no-answer-absent": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + }, + "no-answer-near-miss": { + "queries": 4, + "answerable": 0, + "noAnswerQueries": 4, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 40, + "noAnswerQueryHitRate": 1 + } + } + } + } + }, + "rankings": [ + { + "id": "aurora-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "aurora rolluot rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "aurora rolluot rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10168", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10163", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10158", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10153", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10148", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10143", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10138", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10133", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10128", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + } + ], + "corrections": [ + { + "from": "rolluot", + "to": [ + "rollout" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "aurora roll rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "aurora roll rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10168", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10163", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10158", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10153", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10148", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10143", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10138", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10133", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10128", + "chatId": "999", + "match": "beginnings", + "score": 8.32129250405688 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"aurora rollout rollback\"", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"aurora rollout rollback\"", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "aurora rollout from:700", + "relevant": { + "10001": 2 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": null, + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "aurora rollout from:700", + "relevant": { + "10001": 2 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": "words", + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "aurora rollout chat:101", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "aurora rollout chat:101", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10293", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10288", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10283", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10278", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10273", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10268", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10263", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10258", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10253", + "chatId": "101", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "aurora rollout date:2026-10-02", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": null, + "score": 4.8518902533575785 + }, + { + "id": "10002", + "chatId": "101", + "match": null, + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "aurora rollout after:2026-10-02 before:2026-10-03", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10001", + "chatId": "101", + "match": "words", + "score": 4.8518902533575785 + }, + { + "id": "10002", + "chatId": "101", + "match": "words", + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "aurora AND rollout AND rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "aurora rollout rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "aurora rollout OR rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10298", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10293", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10288", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10283", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10278", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10273", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10268", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10263", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10258", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10253", + "chatId": "101", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "aurora rollout OR rollback", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10168", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10163", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10158", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10153", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10148", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10143", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10138", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10133", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10128", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "aurora deployment reversal", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "aurora deployment reversal", + "relevant": { + "10001": 2, + "10002": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "anyWord", + "score": 8.206683800405353 + }, + { + "id": "10168", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10163", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10158", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10153", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10148", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10143", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10138", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10133", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10128", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "aurora quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "aurora quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10168", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10163", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10158", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10153", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10148", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10143", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10138", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10133", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10128", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "aurora-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "aurora rollout rollback purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "aurora-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "aurora rollout rollback purple", + "relevant": {}, + "hits": [ + { + "id": "10173", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10168", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10163", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10158", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10153", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10148", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10143", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10138", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10133", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10128", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "cedar-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "cedar bakcup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "cedar bakcup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10465", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10460", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10455", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10450", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10445", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10440", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10435", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10430", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10425", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + } + ], + "corrections": [ + { + "from": "bakcup", + "to": [ + "backup" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "cedar back retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "cedar back retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10465", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10460", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10455", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10450", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10445", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10440", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10435", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10430", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10425", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"cedar backup retention\"", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"cedar backup retention\"", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "cedar backup from:700", + "relevant": { + "10303": 2 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "cedar backup from:700", + "relevant": { + "10303": 2 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "cedar backup chat:102", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "cedar backup chat:102", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10595", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10590", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10585", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10580", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10575", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10570", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10565", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10560", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10555", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "cedar backup date:2026-10-02", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10304", + "chatId": "102", + "match": null, + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "cedar backup after:2026-10-02 before:2026-10-03", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10304", + "chatId": "102", + "match": "words", + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "cedar AND backup AND retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "cedar backup retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "cedar backup OR retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10600", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10595", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10590", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10585", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10580", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10575", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10570", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10565", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10560", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10555", + "chatId": "102", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "cedar backup OR retention", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10465", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10460", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10455", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10450", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10445", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10440", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10435", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10430", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10425", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "cedar snapshot lifetime", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "cedar snapshot lifetime", + "relevant": { + "10303": 2, + "10304": 1 + }, + "hits": [ + { + "id": "10303", + "chatId": "102", + "match": "anyWord", + "score": 10.379727637637021 + }, + { + "id": "11511", + "chatId": "106", + "match": "anyWord", + "score": 7.449128259278972 + }, + { + "id": "10465", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10460", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10455", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10450", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10445", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10440", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10435", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10430", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "cedar quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "cedar quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10465", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10460", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10455", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10450", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10445", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10440", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10435", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10430", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10425", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "cedar-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "cedar backup retention purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "cedar-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "cedar backup retention purple", + "relevant": {}, + "hits": [ + { + "id": "10470", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10465", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10460", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10455", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10450", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10445", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10440", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10435", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10430", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10425", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "север-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 8.32129250405688 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 8.32129250405688 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "север сертифкат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "север-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "север сертифкат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "corrected", + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": "corrected", + "score": 8.32129250405688 + }, + { + "id": "10767", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10762", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10757", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10752", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10747", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10742", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10737", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "10732", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + } + ], + "corrections": [ + { + "from": "сертифкат", + "to": [ + "сертификат" + ] + } + ], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "север серти обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "север-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "север серти обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "beginnings", + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": "beginnings", + "score": 8.32129250405688 + }, + { + "id": "10767", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10762", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10757", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10752", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10747", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10742", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10737", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "10732", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"север сертификат обновление\"", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 3.126652600631594 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "10862", + "chatId": "103", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "север-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"север сертификат обновление\"", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 3.126652600631594 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "10727", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "север-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "север сертификат from:700", + "relevant": { + "10605": 2 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 5.824679559744159 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "север сертификат from:700", + "relevant": { + "10605": 2 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 5.824679559744159 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "север сертификат chat:103", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 5.824679559744159 + }, + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 5.651770526756559 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "север сертификат chat:103", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 5.824679559744159 + }, + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 5.651770526756559 + }, + { + "id": "10902", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10897", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10892", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10887", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10882", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10877", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10872", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "10867", + "chatId": "103", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "север сертификат date:2026-10-02", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 5.824679559744159 + }, + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 5.651770526756559 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "север сертификат after:2026-10-02 before:2026-10-03", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 5.824679559744159 + }, + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 5.651770526756559 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "север-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "север AND сертификат AND обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 8.32129250405688 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "север сертификат обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 8.32129250405688 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "север сертификат OR обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": null, + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": null, + "score": 8.32129250405688 + }, + { + "id": "10902", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10897", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10892", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10887", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10882", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10877", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10872", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "10867", + "chatId": "103", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "север сертификат OR обновление", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "words", + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": "words", + "score": 8.32129250405688 + }, + { + "id": "10767", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10762", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10757", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10752", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10747", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10742", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10737", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "10732", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "север-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "север продление TLS", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "север-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "север продление TLS", + "relevant": { + "10605": 2, + "10606": 1 + }, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "anyWord", + "score": 2.8258852633782796 + }, + { + "id": "10767", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10762", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10757", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10752", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10747", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10742", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10737", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10732", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10727", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "север-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "север quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "север-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "север quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "anyWord", + "score": 2.8258852633782796 + }, + { + "id": "10767", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10762", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10757", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10752", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10747", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10742", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10737", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10732", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "10727", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "север-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "север сертификат обновление purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "север-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "север сертификат обновление purple", + "relevant": {}, + "hits": [ + { + "id": "10606", + "chatId": "103", + "match": "anyWord", + "score": 8.47765579013484 + }, + { + "id": "10605", + "chatId": "103", + "match": "anyWord", + "score": 8.32129250405688 + }, + { + "id": "10767", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10762", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10757", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10752", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10747", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10742", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10737", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "10732", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "lumen-words", + "split": "dev", + "category": "words", + "mode": "lucene", + "text": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-words", + "split": "dev", + "category": "words", + "mode": "legacy", + "text": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-typo", + "split": "dev", + "category": "typo", + "mode": "lucene", + "text": "lumen invocie refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-typo", + "split": "dev", + "category": "typo", + "mode": "legacy", + "text": "lumen invocie refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11074", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11069", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11064", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11059", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11054", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11049", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11044", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11039", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11034", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + } + ], + "corrections": [ + { + "from": "invocie", + "to": [ + "invoice" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "lucene", + "text": "lumen invo refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-beginnings", + "split": "dev", + "category": "beginnings", + "mode": "legacy", + "text": "lumen invo refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11074", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11069", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11064", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11059", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11054", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11049", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11044", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11039", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11034", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-phrase", + "split": "dev", + "category": "phrase", + "mode": "lucene", + "text": "\"lumen invoice refund\"", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-phrase", + "split": "dev", + "category": "phrase", + "mode": "legacy", + "text": "\"lumen invoice refund\"", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-from", + "split": "dev", + "category": "from", + "mode": "lucene", + "text": "lumen invoice from:700", + "relevant": { + "10907": 2 + }, + "hits": [ + { + "id": "10907", + "chatId": "104", + "match": null, + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-from", + "split": "dev", + "category": "from", + "mode": "legacy", + "text": "lumen invoice from:700", + "relevant": { + "10907": 2 + }, + "hits": [ + { + "id": "10907", + "chatId": "104", + "match": "words", + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-chat", + "split": "dev", + "category": "chat", + "mode": "lucene", + "text": "lumen invoice chat:104", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-chat", + "split": "dev", + "category": "chat", + "mode": "legacy", + "text": "lumen invoice chat:104", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11199", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11194", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11189", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11184", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11179", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11174", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11169", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11164", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11159", + "chatId": "104", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-date", + "split": "dev", + "category": "date", + "mode": "lucene", + "text": "lumen invoice date:2026-10-02", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "10908", + "chatId": "104", + "match": null, + "score": 4.8518902533575785 + }, + { + "id": "10907", + "chatId": "104", + "match": null, + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-date", + "split": "dev", + "category": "date", + "mode": "legacy", + "text": "lumen invoice after:2026-10-02 before:2026-10-03", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "10908", + "chatId": "104", + "match": "words", + "score": 4.8518902533575785 + }, + { + "id": "10907", + "chatId": "104", + "match": "words", + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-and", + "split": "dev", + "category": "and", + "mode": "lucene", + "text": "lumen AND invoice AND refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-and", + "split": "dev", + "category": "and", + "mode": "legacy", + "text": "lumen invoice refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-or", + "split": "dev", + "category": "or", + "mode": "lucene", + "text": "lumen invoice OR refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11204", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11199", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11194", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11189", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11184", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11179", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11174", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11169", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11164", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11159", + "chatId": "104", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-or", + "split": "dev", + "category": "or", + "mode": "legacy", + "text": "lumen invoice OR refund", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11074", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11069", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11064", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11059", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11054", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11049", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11044", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11039", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11034", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "lucene", + "text": "lumen payment reimbursement", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-paraphrase", + "split": "dev", + "category": "paraphrase", + "mode": "legacy", + "text": "lumen payment reimbursement", + "relevant": { + "10907": 2, + "10908": 1 + }, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11074", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11069", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11064", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11059", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11054", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11049", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11044", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11039", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11034", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "lucene", + "text": "lumen quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-absent", + "split": "dev", + "category": "no-answer-absent", + "mode": "legacy", + "text": "lumen quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11074", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11069", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11064", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11059", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11054", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11049", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11044", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11039", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11034", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "lumen-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "lumen invoice refund purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "lumen-near-miss", + "split": "dev", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "lumen invoice refund purple", + "relevant": {}, + "hits": [ + { + "id": "11079", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11074", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11069", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11064", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11059", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11054", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11049", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11044", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11039", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11034", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "harbor-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "harbor relesae freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "harbor relesae freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11376", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11371", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11366", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11361", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11356", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11351", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11346", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11341", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11336", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + } + ], + "corrections": [ + { + "from": "relesae", + "to": [ + "release" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "harbor rele freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 9.174703389680088 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "harbor rele freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11376", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11371", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11366", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11361", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11356", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11351", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11346", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11341", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11336", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"harbor release freeze\"", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"harbor release freeze\"", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "harbor release from:700", + "relevant": { + "11209": 2 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": null, + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "harbor release from:700", + "relevant": { + "11209": 2 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "words", + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "harbor release chat:105", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "harbor release chat:105", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11501", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11496", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11491", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11486", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11481", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11476", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11471", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11466", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11461", + "chatId": "105", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "harbor release date:2026-10-02", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": null, + "score": 4.8518902533575785 + }, + { + "id": "11210", + "chatId": "105", + "match": null, + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "harbor release after:2026-10-02 before:2026-10-03", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "words", + "score": 4.8518902533575785 + }, + { + "id": "11210", + "chatId": "105", + "match": "words", + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "harbor AND release AND freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "harbor release freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "harbor release OR freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11501", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11496", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11491", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11486", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11481", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11476", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11471", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11466", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "11461", + "chatId": "105", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "harbor release OR freeze", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11376", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11371", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11366", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11361", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11356", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11351", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11346", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11341", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11336", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "harbor shipping pause", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "harbor shipping pause", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "chatId": "105", + "match": "anyWord", + "score": 10.632628927084141 + }, + { + "id": "11376", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11371", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11366", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11361", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11356", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11351", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11346", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11341", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11336", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [ + { + "from": "pause", + "to": [ + "passes" + ] + } + ], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "harbor quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "harbor quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11376", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11371", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11366", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11361", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11356", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11351", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11346", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11341", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11336", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "harbor-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "harbor release freeze purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "harbor-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "harbor release freeze purple", + "relevant": {}, + "hits": [ + { + "id": "11381", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11376", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11371", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11366", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11361", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11356", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11351", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11346", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11341", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11336", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "maple-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "maple arhcive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "maple arhcive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11678", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11673", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11668", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11663", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11658", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11653", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11648", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11643", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + }, + { + "id": "11638", + "chatId": "999", + "match": "corrected", + "score": 7.48582930091083 + } + ], + "corrections": [ + { + "from": "arhcive", + "to": [ + "archive" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "maple arch restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "maple arch restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11678", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11673", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11668", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11663", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11658", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11653", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11648", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11643", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + }, + { + "id": "11638", + "chatId": "999", + "match": "beginnings", + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"maple archive restore\"", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"maple archive restore\"", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "maple archive from:700", + "relevant": { + "11511": 2 + }, + "hits": [ + { + "id": "11511", + "chatId": "106", + "match": null, + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "maple archive from:700", + "relevant": { + "11511": 2 + }, + "hits": [ + { + "id": "11511", + "chatId": "106", + "match": "words", + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "maple archive chat:106", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "maple archive chat:106", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11803", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11798", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11793", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11788", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11783", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11778", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11773", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11768", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "11763", + "chatId": "106", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "maple archive date:2026-10-02", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11512", + "chatId": "106", + "match": null, + "score": 4.718335520783594 + }, + { + "id": "11511", + "chatId": "106", + "match": null, + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "maple archive after:2026-10-02 before:2026-10-03", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11512", + "chatId": "106", + "match": "words", + "score": 4.718335520783594 + }, + { + "id": "11511", + "chatId": "106", + "match": "words", + "score": 4.718335520783594 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "maple AND archive AND restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "maple archive restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "maple archive OR restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11803", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11798", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11793", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11788", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11783", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11778", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11773", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11768", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + }, + { + "id": "11763", + "chatId": "106", + "match": null, + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "maple archive OR restore", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11678", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11673", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11668", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11663", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11658", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11653", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11648", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11643", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + }, + { + "id": "11638", + "chatId": "999", + "match": "words", + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "maple recover saved data", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "maple recover saved data", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11511", + "chatId": "106", + "match": "anyWord", + "score": 10.33995167373274 + }, + { + "id": "11209", + "chatId": "105", + "match": "anyWord", + "score": 8.206683800405353 + }, + { + "id": "11673", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11668", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11663", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11658", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11653", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11648", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11643", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11638", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "maple quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "maple quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11678", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11673", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11668", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11663", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11658", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11653", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11648", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11643", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11638", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "maple-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "maple archive restore purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "maple-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "maple archive restore purple", + "relevant": {}, + "hits": [ + { + "id": "11683", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11678", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11673", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11668", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11663", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11658", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11653", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11648", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11643", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + }, + { + "id": "11638", + "chatId": "999", + "match": "anyWord", + "score": 7.48582930091083 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "вектор-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 8.47765579013484 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 8.47765579013484 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "вектор миграця отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + }, + { + "id": "12065", + "chatId": "107", + "match": null, + "score": 10.005267331328891 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "вектор миграця отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "corrected", + "score": 8.47765579013484 + }, + { + "id": "11975", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11970", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11965", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11960", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11955", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11950", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11945", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11940", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "11935", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + } + ], + "corrections": [ + { + "from": "миграця", + "to": [ + "миграция" + ] + } + ], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "вектор мигр отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "вектор мигр отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "beginnings", + "score": 8.47765579013484 + }, + { + "id": "11975", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11970", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11965", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11960", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11955", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11950", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11945", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11940", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "11935", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"вектор миграция отмена\"", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 3.126652600631594 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"вектор миграция отмена\"", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 3.126652600631594 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "вектор миграция from:700", + "relevant": { + "11813": 2 + }, + "hits": [ + { + "id": "11813", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "вектор миграция from:700", + "relevant": { + "11813": 2 + }, + "hits": [ + { + "id": "11813", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "вектор миграция chat:107", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 5.651770526756559 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "вектор миграция chat:107", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 5.651770526756559 + }, + { + "id": "12110", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12105", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12100", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12095", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12090", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12085", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12080", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12075", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12070", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "вектор миграция date:2026-10-02", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 5.651770526756559 + }, + { + "id": "11813", + "chatId": "107", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "вектор миграция after:2026-10-02 before:2026-10-03", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 5.651770526756559 + }, + { + "id": "11813", + "chatId": "107", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "вектор AND миграция AND отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 8.47765579013484 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "вектор миграция отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 8.47765579013484 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "вектор миграция OR отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": null, + "score": 8.47765579013484 + }, + { + "id": "12110", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12105", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12100", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12095", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12090", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12085", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12080", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12075", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12070", + "chatId": "107", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "вектор миграция OR отмена", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "words", + "score": 8.47765579013484 + }, + { + "id": "11975", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11970", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11965", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11960", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11955", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11950", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11945", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11940", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "11935", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "вектор возврат прежней схемы", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "вектор возврат прежней схемы", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11813", + "chatId": "107", + "match": "anyWord", + "score": 10.942357565924837 + }, + { + "id": "11814", + "chatId": "107", + "match": "anyWord", + "score": 2.8258852633782796 + }, + { + "id": "11975", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11970", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11965", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11960", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11955", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11950", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11945", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11940", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [ + { + "from": "схемы", + "to": [ + "схему" + ] + } + ], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "вектор quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "вектор quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "anyWord", + "score": 2.8258852633782796 + }, + { + "id": "11975", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11970", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11965", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11960", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11955", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11950", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11945", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11940", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "11935", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "вектор-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "вектор миграция отмена purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "вектор-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "вектор миграция отмена purple", + "relevant": {}, + "hits": [ + { + "id": "11814", + "chatId": "107", + "match": "anyWord", + "score": 8.47765579013484 + }, + { + "id": "11975", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11970", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11965", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11960", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11955", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11950", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11945", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11940", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "11935", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "prism-words", + "split": "test", + "category": "words", + "mode": "lucene", + "text": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-words", + "split": "test", + "category": "words", + "mode": "legacy", + "text": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-typo", + "split": "test", + "category": "typo", + "mode": "lucene", + "text": "prism renweal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-typo", + "split": "test", + "category": "typo", + "mode": "legacy", + "text": "prism renweal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12282", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12277", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12272", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12267", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12262", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12257", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12252", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12247", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + }, + { + "id": "12242", + "chatId": "999", + "match": "corrected", + "score": 7.489838832938161 + } + ], + "corrections": [ + { + "from": "renweal", + "to": [ + "renewal" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-beginnings", + "split": "test", + "category": "beginnings", + "mode": "lucene", + "text": "prism rene discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-beginnings", + "split": "test", + "category": "beginnings", + "mode": "legacy", + "text": "prism rene discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12282", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12277", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12272", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12267", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12262", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12257", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12252", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12247", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + }, + { + "id": "12242", + "chatId": "999", + "match": "beginnings", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-phrase", + "split": "test", + "category": "phrase", + "mode": "lucene", + "text": "\"prism renewal discount\"", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-phrase", + "split": "test", + "category": "phrase", + "mode": "legacy", + "text": "\"prism renewal discount\"", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 2.7623348535297314 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-from", + "split": "test", + "category": "from", + "mode": "lucene", + "text": "prism renewal from:700", + "relevant": { + "12115": 2 + }, + "hits": [ + { + "id": "12115", + "chatId": "108", + "match": null, + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-from", + "split": "test", + "category": "from", + "mode": "legacy", + "text": "prism renewal from:700", + "relevant": { + "12115": 2 + }, + "hits": [ + { + "id": "12115", + "chatId": "108", + "match": "words", + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-chat", + "split": "test", + "category": "chat", + "mode": "lucene", + "text": "prism renewal chat:108", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-chat", + "split": "test", + "category": "chat", + "mode": "legacy", + "text": "prism renewal chat:108", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12407", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12402", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12397", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12392", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12387", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12382", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12377", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12372", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + }, + { + "id": "12367", + "chatId": "108", + "match": "words", + "score": 4.993225888625441 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-date", + "split": "test", + "category": "date", + "mode": "lucene", + "text": "prism renewal date:2026-10-02", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12116", + "chatId": "108", + "match": null, + "score": 4.8518902533575785 + }, + { + "id": "12115", + "chatId": "108", + "match": null, + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-date", + "split": "test", + "category": "date", + "mode": "legacy", + "text": "prism renewal after:2026-10-02 before:2026-10-03", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12116", + "chatId": "108", + "match": "words", + "score": 4.8518902533575785 + }, + { + "id": "12115", + "chatId": "108", + "match": "words", + "score": 4.8518902533575785 + } + ], + "corrections": [], + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-and", + "split": "test", + "category": "and", + "mode": "lucene", + "text": "prism AND renewal AND discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-and", + "split": "test", + "category": "and", + "mode": "legacy", + "text": "prism renewal discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-or", + "split": "test", + "category": "or", + "mode": "lucene", + "text": "prism renewal OR discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12407", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12402", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12397", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12392", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12387", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12382", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12377", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12372", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + }, + { + "id": "12367", + "chatId": "108", + "match": null, + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-or", + "split": "test", + "category": "or", + "mode": "legacy", + "text": "prism renewal OR discount", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12282", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12277", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12272", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12267", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12262", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12257", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12252", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12247", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + }, + { + "id": "12242", + "chatId": "999", + "match": "words", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "lucene", + "text": "prism subscription price reduction", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [], + "corrections": [], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-paraphrase", + "split": "test", + "category": "paraphrase", + "mode": "legacy", + "text": "prism subscription price reduction", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12282", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12277", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12272", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12267", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12262", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12257", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12252", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12247", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + }, + { + "id": "12242", + "chatId": "999", + "match": "anyWord", + "score": 4.993225888625441 + } + ], + "corrections": [ + { + "from": "price", + "to": [ + "prism" + ] + } + ], + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "lucene", + "text": "prism quantum unicorn", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-absent", + "split": "test", + "category": "no-answer-absent", + "mode": "legacy", + "text": "prism quantum unicorn", + "relevant": {}, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12282", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12277", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12272", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12267", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12262", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12257", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12252", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12247", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + }, + { + "id": "12242", + "chatId": "999", + "match": "anyWord", + "score": 2.4966129443127203 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + }, + { + "id": "prism-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "lucene", + "text": "prism renewal discount purple", + "relevant": {}, + "hits": [], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 0 + }, + { + "id": "prism-near-miss", + "split": "test", + "category": "no-answer-near-miss", + "mode": "legacy", + "text": "prism renewal discount purple", + "relevant": {}, + "hits": [ + { + "id": "12287", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12282", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12277", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12272", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12267", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12262", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12257", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12252", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12247", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + }, + { + "id": "12242", + "chatId": "999", + "match": "anyWord", + "score": 7.489838832938161 + } + ], + "corrections": [], + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "noAnswerFalseHits": 10 + } + ] +} diff --git a/bench/message-search-quality/crossencoder-dev.json b/bench/message-search-quality/crossencoder-dev.json new file mode 100644 index 000000000..d452cc3b5 --- /dev/null +++ b/bench/message-search-quality/crossencoder-dev.json @@ -0,0 +1,30187 @@ +{ + "crossEncoderImplementationSha256": "2d044b35dc1f4ac769127e1b86303b5b4d20f7970eb2ea9a47df729153e96aa0", + "rerankerManifestSha256": "8efb505319d19cb42772dc02438cd99c68cfd4771bcba8890aac3ec81b1538d3", + "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", + "questionsSha256": "390ff1475964d982a06afaf9e2f6f840f5b8304fcffe58ebedd5a0ea8d54478e", + "runnerSha256": "bea007961d0dc1d237824c274029f81dce28c4840e8df88c9b2b282d9bf07fd5", + "productionSha256": { + "services/messages-combined.js": "d10aeb5f7049f615503d59c37df109bcd284f53b5cfceaac0151f2d7e2ca60e9", + "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", + "embeddings/embed.js": "cacf0bec8b98aeda4ed135642d593c4d5b50800867bc8e95de0ad77c1e002e3f", + "embeddings/models.js": "8debfaa01509728ba206702f27e6d8251b8968a03375089b6ca1af9d0eb53e94", + "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964" + }, + "node": "v24.19.0", + "cpu": "AMD Ryzen AI 9 HX 470 w/ Radeon 890M", + "storePath": "/tmp/semantic-message-quality-rDgMNR/quality.db", + "model": { + "id": "cross-encoder/mmarco-mMiniLMv2-L12-H384-v1", + "revision": "1427fd652930e4ba29e8149678df786c240d8825", + "licence": "Apache-2.0", + "files": [ + { + "name": "onnx/model_quint8_avx2.onnx", + "sha256": "6c2513767fb63d008a4377bef7a7a3555433d9436342bb53e35a3a72ffc52d4b" + }, + { + "name": "tokenizer.json", + "sha256": "62c24cdc13d4c9952d63718d6c9fa4c287974249e16b7ade6d5a85e7bbb75626" + }, + { + "name": "tokenizer_config.json", + "sha256": "e7fbfbfa6347b4e414c1cee50d142e2c2f9a895dad68b068ae83a8b564c3837e" + }, + { + "name": "config.json", + "sha256": "cc2cfe51aa3fd759d21d21acf5dfd6994aa67a3c9210636d22e143699d336c77" + } + ], + "threads": 8 + }, + "kind": "cross-encoder raw logit reranking", + "candidateSummary": [ + { + "depth": 50, + "candidateRecall": 0.4375, + "byCategory": { + "words": 0.25, + "typo": 0.25, + "beginnings": 0.25, + "phrase": 0.125, + "from": 1, + "chat": 1, + "date": 1, + "and": 0.25, + "or": 0.25, + "paraphrase": 0 + } + }, + { + "depth": 100, + "candidateRecall": 0.75, + "byCategory": { + "words": 0.75, + "typo": 0.75, + "beginnings": 0.75, + "phrase": 0.75, + "from": 1, + "chat": 1, + "date": 1, + "and": 0.75, + "or": 0.75, + "paraphrase": 0 + } + }, + { + "depth": 300, + "candidateRecall": 0.9, + "byCategory": { + "words": 1, + "typo": 1, + "beginnings": 1, + "phrase": 1, + "from": 1, + "chat": 1, + "date": 1, + "and": 1, + "or": 1, + "paraphrase": 0 + } + }, + { + "depth": 500, + "candidateRecall": 0.9, + "byCategory": { + "words": 1, + "typo": 1, + "beginnings": 1, + "phrase": 1, + "from": 1, + "chat": 1, + "date": 1, + "and": 1, + "or": 1, + "paraphrase": 0 + } + } + ], + "selectedOnDev": { + "depth": 300, + "questionScoreFloor": 2 + }, + "keywordSummary": { + "lexical": { + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0 + }, + "semantic": { + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.525, + "mrr": 0.553125, + "ndcgAt10": 0.4769238266733387, + "noAnswerFalseHits": 0 + } + }, + "questionSummary": { + "direct": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + "anchoredLexical": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 40 + }, + "sweep": [ + { + "floor": null, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.6875, + "mrr": 0.875, + "ndcgAt10": 0.7735661101175583, + "noAnswerFalseHits": 40 + } + }, + { + "floor": -5, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.6875, + "mrr": 0.875, + "ndcgAt10": 0.7735661101175583, + "noAnswerFalseHits": 40 + } + }, + { + "floor": -2, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.625, + "mrr": 0.875, + "ndcgAt10": 0.7627057761880933, + "noAnswerFalseHits": 13 + } + }, + { + "floor": -1, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.625, + "mrr": 0.875, + "ndcgAt10": 0.7627057761880933, + "noAnswerFalseHits": 6 + } + }, + { + "floor": 0, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.625, + "mrr": 0.875, + "ndcgAt10": 0.7627057761880933, + "noAnswerFalseHits": 3 + } + }, + { + "floor": 1, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.875, + "ndcgAt10": 0.7409851083291633, + "noAnswerFalseHits": 1 + } + }, + { + "floor": 2, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.875, + "ndcgAt10": 0.7409851083291633, + "noAnswerFalseHits": 0 + } + }, + { + "floor": 3, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.875, + "ndcgAt10": 0.7409851083291633, + "noAnswerFalseHits": 0 + } + }, + { + "floor": 5, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.875, + "ndcgAt10": 0.7409851083291633, + "noAnswerFalseHits": 0 + } + }, + { + "floor": 8, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.25, + "mrr": 0.5, + "ndcgAt10": 0.41311732856427996, + "noAnswerFalseHits": 0 + } + } + ] + }, + "devQualified": true, + "questionRetrieval": "manual topic anchors for diagnostic reranking; full-question direct retrieval is measured separately", + "heldOut": { + "keyword": { + "lexical": { + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.275, + "mrr": 0.35, + "ndcgAt10": 0.226064801430716, + "noAnswerFalseHits": 0 + }, + "semantic": { + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.4375, + "mrr": 0.4875, + "ndcgAt10": 0.40487848841280566, + "noAnswerFalseHits": 0 + }, + "rows": [ + { + "id": "harbor-words", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11210", + "score": 9.45776081085205, + "lexicalRank": 121 + }, + { + "id": "11209", + "score": 9.25239086151123, + "lexicalRank": 60 + }, + { + "id": "11459", + "score": 8.542092323303223, + "lexicalRank": 252 + }, + { + "id": "11319", + "score": 8.532307624816895, + "lexicalRank": 280 + }, + { + "id": "11449", + "score": 8.469769477844238, + "lexicalRank": 254 + }, + { + "id": "11409", + "score": 8.397802352905273, + "lexicalRank": 262 + }, + { + "id": "11439", + "score": 8.30823802947998, + "lexicalRank": 256 + }, + { + "id": "11344", + "score": 8.291367530822754, + "lexicalRank": 275 + }, + { + "id": "11509", + "score": 8.277020454406738, + "lexicalRank": 242 + }, + { + "id": "11419", + "score": 8.27120304107666, + "lexicalRank": 260 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-typo", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "score": 8.698358535766602, + "lexicalRank": 60 + }, + { + "id": "11210", + "score": 8.513386726379395, + "lexicalRank": 121 + }, + { + "id": "11481", + "score": 7.69588041305542, + "lexicalRank": 5 + }, + { + "id": "11446", + "score": 7.607393741607666, + "lexicalRank": 12 + }, + { + "id": "11346", + "score": 7.577287673950195, + "lexicalRank": 32 + }, + { + "id": "11256", + "score": 7.526706218719482, + "lexicalRank": 50 + }, + { + "id": "11391", + "score": 7.5081915855407715, + "lexicalRank": 23 + }, + { + "id": "11451", + "score": 7.493976593017578, + "lexicalRank": 11 + }, + { + "id": "11381", + "score": 7.487243175506592, + "lexicalRank": 25 + }, + { + "id": "11501", + "score": 7.483785152435303, + "lexicalRank": 1 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-beginnings", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11209", + "score": 9.069320678710938, + "lexicalRank": 60 + }, + { + "id": "11210", + "score": 8.7719144821167, + "lexicalRank": 121 + }, + { + "id": "11301", + "score": 7.857519626617432, + "lexicalRank": 41 + }, + { + "id": "11416", + "score": 7.742708683013916, + "lexicalRank": 18 + }, + { + "id": "11366", + "score": 7.652957916259766, + "lexicalRank": 28 + }, + { + "id": "11211", + "score": 7.649582862854004, + "lexicalRank": 59 + }, + { + "id": "11291", + "score": 7.621363639831543, + "lexicalRank": 43 + }, + { + "id": "11321", + "score": 7.560271263122559, + "lexicalRank": 37 + }, + { + "id": "11346", + "score": 7.559664249420166, + "lexicalRank": 32 + }, + { + "id": "11431", + "score": 7.4946208000183105, + "lexicalRank": 15 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-phrase", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "score": 2.7623348535297314 + }, + { + "id": "11501", + "score": 2.7623348535297314 + }, + { + "id": "11496", + "score": 2.7623348535297314 + }, + { + "id": "11491", + "score": 2.7623348535297314 + }, + { + "id": "11486", + "score": 2.7623348535297314 + }, + { + "id": "11481", + "score": 2.7623348535297314 + }, + { + "id": "11476", + "score": 2.7623348535297314 + }, + { + "id": "11471", + "score": 2.7623348535297314 + }, + { + "id": "11466", + "score": 2.7623348535297314 + }, + { + "id": "11461", + "score": 2.7623348535297314 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-from", + "relevant": { + "11209": 2 + }, + "hits": [ + { + "id": "11209", + "score": -2.535362720489502, + "lexicalRank": 0 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + } + }, + { + "id": "harbor-chat", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11454", + "score": -0.74288409948349, + "lexicalRank": 93 + }, + { + "id": "11494", + "score": -0.8699719309806824, + "lexicalRank": 85 + }, + { + "id": "11414", + "score": -0.9635332226753235, + "lexicalRank": 101 + }, + { + "id": "11424", + "score": -1.1062660217285156, + "lexicalRank": 99 + }, + { + "id": "11444", + "score": -1.1148967742919922, + "lexicalRank": 95 + }, + { + "id": "11434", + "score": -1.1507675647735596, + "lexicalRank": 97 + }, + { + "id": "11489", + "score": -1.187790036201477, + "lexicalRank": 86 + }, + { + "id": "11504", + "score": -1.1919387578964233, + "lexicalRank": 83 + }, + { + "id": "11479", + "score": -1.2202425003051758, + "lexicalRank": 88 + }, + { + "id": "11429", + "score": -1.2790064811706543, + "lexicalRank": 98 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-date", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11210", + "score": -3.488126754760742, + "lexicalRank": 1 + }, + { + "id": "11209", + "score": -3.87473201751709, + "lexicalRank": 0 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + } + }, + { + "id": "harbor-and", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "score": 7.489838832938161 + }, + { + "id": "11501", + "score": 7.489838832938161 + }, + { + "id": "11496", + "score": 7.489838832938161 + }, + { + "id": "11491", + "score": 7.489838832938161 + }, + { + "id": "11486", + "score": 7.489838832938161 + }, + { + "id": "11481", + "score": 7.489838832938161 + }, + { + "id": "11476", + "score": 7.489838832938161 + }, + { + "id": "11471", + "score": 7.489838832938161 + }, + { + "id": "11466", + "score": 7.489838832938161 + }, + { + "id": "11461", + "score": 7.489838832938161 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-or", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [ + { + "id": "11506", + "score": 7.489838832938161 + }, + { + "id": "11501", + "score": 7.489838832938161 + }, + { + "id": "11496", + "score": 7.489838832938161 + }, + { + "id": "11491", + "score": 7.489838832938161 + }, + { + "id": "11486", + "score": 7.489838832938161 + }, + { + "id": "11481", + "score": 7.489838832938161 + }, + { + "id": "11476", + "score": 7.489838832938161 + }, + { + "id": "11471", + "score": 7.489838832938161 + }, + { + "id": "11466", + "score": 7.489838832938161 + }, + { + "id": "11461", + "score": 7.489838832938161 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-paraphrase", + "relevant": { + "11209": 2, + "11210": 1 + }, + "hits": [], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "harbor-absent", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "harbor-near-miss", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "maple-words", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11593", + "score": 7.788802146911621, + "lexicalRank": 43 + }, + { + "id": "11656", + "score": 7.6396074295043945, + "lexicalRank": 273 + }, + { + "id": "11671", + "score": 7.6235737800598145, + "lexicalRank": 270 + }, + { + "id": "11751", + "score": 7.563724517822266, + "lexicalRank": 254 + }, + { + "id": "11553", + "score": 7.5539326667785645, + "lexicalRank": 51 + }, + { + "id": "11768", + "score": 7.544630527496338, + "lexicalRank": 8 + }, + { + "id": "11716", + "score": 7.518007278442383, + "lexicalRank": 261 + }, + { + "id": "11756", + "score": 7.508123874664307, + "lexicalRank": 253 + }, + { + "id": "11701", + "score": 7.493647575378418, + "lexicalRank": 264 + }, + { + "id": "11653", + "score": 7.427809715270996, + "lexicalRank": 31 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-typo", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11511", + "score": 4.387134075164795, + "lexicalRank": 121 + }, + { + "id": "11593", + "score": 4.083454608917236, + "lexicalRank": 43 + }, + { + "id": "11693", + "score": 3.98818302154541, + "lexicalRank": 23 + }, + { + "id": "11708", + "score": 3.9621403217315674, + "lexicalRank": 20 + }, + { + "id": "11788", + "score": 3.935478687286377, + "lexicalRank": 4 + }, + { + "id": "11738", + "score": 3.922764539718628, + "lexicalRank": 14 + }, + { + "id": "11803", + "score": 3.8401293754577637, + "lexicalRank": 1 + }, + { + "id": "11763", + "score": 3.818089485168457, + "lexicalRank": 9 + }, + { + "id": "11798", + "score": 3.810894250869751, + "lexicalRank": 2 + }, + { + "id": "11768", + "score": 3.8016297817230225, + "lexicalRank": 8 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-beginnings", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11512", + "score": 4.051966190338135, + "lexicalRank": 120 + }, + { + "id": "11633", + "score": 3.726194381713867, + "lexicalRank": 35 + }, + { + "id": "11717", + "score": 3.639094829559326, + "lexicalRank": 201 + }, + { + "id": "11511", + "score": 3.412858009338379, + "lexicalRank": 121 + }, + { + "id": "11618", + "score": 3.4080142974853516, + "lexicalRank": 38 + }, + { + "id": "11738", + "score": 3.2612063884735107, + "lexicalRank": 14 + }, + { + "id": "11538", + "score": 3.1791515350341797, + "lexicalRank": 54 + }, + { + "id": "11767", + "score": 3.1666500568389893, + "lexicalRank": 191 + }, + { + "id": "11773", + "score": 3.1325058937072754, + "lexicalRank": 7 + }, + { + "id": "11798", + "score": 3.0707685947418213, + "lexicalRank": 2 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.631251450669265, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-phrase", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "score": 2.7623348535297314 + }, + { + "id": "11803", + "score": 2.7623348535297314 + }, + { + "id": "11798", + "score": 2.7623348535297314 + }, + { + "id": "11793", + "score": 2.7623348535297314 + }, + { + "id": "11788", + "score": 2.7623348535297314 + }, + { + "id": "11783", + "score": 2.7623348535297314 + }, + { + "id": "11778", + "score": 2.7623348535297314 + }, + { + "id": "11773", + "score": 2.7623348535297314 + }, + { + "id": "11768", + "score": 2.7623348535297314 + }, + { + "id": "11763", + "score": 2.7623348535297314 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-from", + "relevant": { + "11511": 2 + }, + "hits": [ + { + "id": "11511", + "score": -3.522339105606079, + "lexicalRank": 0 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + } + }, + { + "id": "maple-chat", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11736", + "score": -1.9755055904388428, + "lexicalRank": 97 + }, + { + "id": "11716", + "score": -2.040720224380493, + "lexicalRank": 101 + }, + { + "id": "11788", + "score": -2.0611703395843506, + "lexicalRank": 4 + }, + { + "id": "11793", + "score": -2.237440347671509, + "lexicalRank": 3 + }, + { + "id": "11721", + "score": -2.247375249862671, + "lexicalRank": 100 + }, + { + "id": "11761", + "score": -2.2613048553466797, + "lexicalRank": 92 + }, + { + "id": "11726", + "score": -2.2749173641204834, + "lexicalRank": 99 + }, + { + "id": "11801", + "score": -2.2842514514923096, + "lexicalRank": 84 + }, + { + "id": "11771", + "score": -2.3484065532684326, + "lexicalRank": 90 + }, + { + "id": "11746", + "score": -2.3657286167144775, + "lexicalRank": 95 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-date", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11511", + "score": -3.7387211322784424, + "lexicalRank": 1 + }, + { + "id": "11512", + "score": -3.991640329360962, + "lexicalRank": 0 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + } + }, + { + "id": "maple-and", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "score": 7.48582930091083 + }, + { + "id": "11803", + "score": 7.48582930091083 + }, + { + "id": "11798", + "score": 7.48582930091083 + }, + { + "id": "11793", + "score": 7.48582930091083 + }, + { + "id": "11788", + "score": 7.48582930091083 + }, + { + "id": "11783", + "score": 7.48582930091083 + }, + { + "id": "11778", + "score": 7.48582930091083 + }, + { + "id": "11773", + "score": 7.48582930091083 + }, + { + "id": "11768", + "score": 7.48582930091083 + }, + { + "id": "11763", + "score": 7.48582930091083 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-or", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [ + { + "id": "11808", + "score": 7.48582930091083 + }, + { + "id": "11803", + "score": 7.48582930091083 + }, + { + "id": "11798", + "score": 7.48582930091083 + }, + { + "id": "11793", + "score": 7.48582930091083 + }, + { + "id": "11788", + "score": 7.48582930091083 + }, + { + "id": "11783", + "score": 7.48582930091083 + }, + { + "id": "11778", + "score": 7.48582930091083 + }, + { + "id": "11773", + "score": 7.48582930091083 + }, + { + "id": "11768", + "score": 7.48582930091083 + }, + { + "id": "11763", + "score": 7.48582930091083 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-paraphrase", + "relevant": { + "11511": 2, + "11512": 1 + }, + "hits": [], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "maple-absent", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "maple-near-miss", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "вектор-words", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11940", + "score": 7.931367874145508, + "lexicalRank": 35 + }, + { + "id": "11835", + "score": 7.92194128036499, + "lexicalRank": 56 + }, + { + "id": "11945", + "score": 7.5705389976501465, + "lexicalRank": 34 + }, + { + "id": "12015", + "score": 7.566339015960693, + "lexicalRank": 20 + }, + { + "id": "11910", + "score": 7.426838397979736, + "lexicalRank": 41 + }, + { + "id": "11990", + "score": 7.4217610359191895, + "lexicalRank": 25 + }, + { + "id": "11830", + "score": 7.3859992027282715, + "lexicalRank": 57 + }, + { + "id": "11915", + "score": 7.373218059539795, + "lexicalRank": 40 + }, + { + "id": "12075", + "score": 7.369237899780273, + "lexicalRank": 8 + }, + { + "id": "12035", + "score": 7.345060348510742, + "lexicalRank": 16 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + } + }, + { + "id": "вектор-typo", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11995", + "score": 6.678157329559326, + "lexicalRank": 24 + }, + { + "id": "11813", + "score": 6.626920223236084, + "lexicalRank": 101 + }, + { + "id": "12075", + "score": 6.614440441131592, + "lexicalRank": 7 + }, + { + "id": "11840", + "score": 6.4451212882995605, + "lexicalRank": 55 + }, + { + "id": "11970", + "score": 6.295319557189941, + "lexicalRank": 29 + }, + { + "id": "11890", + "score": 6.290195465087891, + "lexicalRank": 45 + }, + { + "id": "12040", + "score": 6.255374431610107, + "lexicalRank": 14 + }, + { + "id": "11820", + "score": 6.2226386070251465, + "lexicalRank": 59 + }, + { + "id": "11934", + "score": 6.207514762878418, + "lexicalRank": 218 + }, + { + "id": "12060", + "score": 6.2029337882995605, + "lexicalRank": 10 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 0.5, + "ndcgAt10": 0.52129602861432, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "вектор-beginnings", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11989", + "score": 3.2008016109466553, + "lexicalRank": 207 + }, + { + "id": "12019", + "score": 3.1179652214050293, + "lexicalRank": 201 + }, + { + "id": "11969", + "score": 3.0732762813568115, + "lexicalRank": 211 + }, + { + "id": "11889", + "score": 2.985116481781006, + "lexicalRank": 227 + }, + { + "id": "12004", + "score": 2.9849021434783936, + "lexicalRank": 204 + }, + { + "id": "12029", + "score": 2.914386749267578, + "lexicalRank": 199 + }, + { + "id": "11939", + "score": 2.819045066833496, + "lexicalRank": 217 + }, + { + "id": "11904", + "score": 2.8098416328430176, + "lexicalRank": 224 + }, + { + "id": "11924", + "score": 2.808605909347534, + "lexicalRank": 220 + }, + { + "id": "11934", + "score": 2.789806365966797, + "lexicalRank": 218 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + } + }, + { + "id": "вектор-phrase", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "score": 3.126652600631594 + }, + { + "id": "12110", + "score": 2.7623348535297314 + }, + { + "id": "12105", + "score": 2.7623348535297314 + }, + { + "id": "12100", + "score": 2.7623348535297314 + }, + { + "id": "12095", + "score": 2.7623348535297314 + }, + { + "id": "12090", + "score": 2.7623348535297314 + }, + { + "id": "12085", + "score": 2.7623348535297314 + }, + { + "id": "12080", + "score": 2.7623348535297314 + }, + { + "id": "12075", + "score": 2.7623348535297314 + }, + { + "id": "12070", + "score": 2.7623348535297314 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + } + }, + { + "id": "вектор-from", + "relevant": { + "11813": 2 + }, + "hits": [ + { + "id": "11813", + "score": -4.053653717041016, + "lexicalRank": 0 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + } + }, + { + "id": "вектор-chat", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "12043", + "score": -0.7538442611694336, + "lexicalRank": 96 + }, + { + "id": "12033", + "score": -0.8733174204826355, + "lexicalRank": 98 + }, + { + "id": "12028", + "score": -0.9477561116218567, + "lexicalRank": 99 + }, + { + "id": "12023", + "score": -1.0663375854492188, + "lexicalRank": 100 + }, + { + "id": "12083", + "score": -1.1414753198623657, + "lexicalRank": 88 + }, + { + "id": "12018", + "score": -1.2144559621810913, + "lexicalRank": 101 + }, + { + "id": "12038", + "score": -1.4278435707092285, + "lexicalRank": 97 + }, + { + "id": "12103", + "score": -1.4810831546783447, + "lexicalRank": 84 + }, + { + "id": "12068", + "score": -1.4991815090179443, + "lexicalRank": 91 + }, + { + "id": "12113", + "score": -1.5094108581542969, + "lexicalRank": 82 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + } + }, + { + "id": "вектор-date", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11813", + "score": -5.503037452697754, + "lexicalRank": 1 + }, + { + "id": "11814", + "score": -5.587318420410156, + "lexicalRank": 0 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + } + }, + { + "id": "вектор-and", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "score": 8.47765579013484 + }, + { + "id": "12110", + "score": 7.489838832938161 + }, + { + "id": "12105", + "score": 7.489838832938161 + }, + { + "id": "12100", + "score": 7.489838832938161 + }, + { + "id": "12095", + "score": 7.489838832938161 + }, + { + "id": "12090", + "score": 7.489838832938161 + }, + { + "id": "12085", + "score": 7.489838832938161 + }, + { + "id": "12080", + "score": 7.489838832938161 + }, + { + "id": "12075", + "score": 7.489838832938161 + }, + { + "id": "12070", + "score": 7.489838832938161 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + } + }, + { + "id": "вектор-or", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [ + { + "id": "11814", + "score": 8.47765579013484 + }, + { + "id": "12110", + "score": 7.489838832938161 + }, + { + "id": "12105", + "score": 7.489838832938161 + }, + { + "id": "12100", + "score": 7.489838832938161 + }, + { + "id": "12095", + "score": 7.489838832938161 + }, + { + "id": "12090", + "score": 7.489838832938161 + }, + { + "id": "12085", + "score": 7.489838832938161 + }, + { + "id": "12080", + "score": 7.489838832938161 + }, + { + "id": "12075", + "score": 7.489838832938161 + }, + { + "id": "12070", + "score": 7.489838832938161 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + } + }, + { + "id": "вектор-paraphrase", + "relevant": { + "11813": 2, + "11814": 1 + }, + "hits": [], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "вектор-absent", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "вектор-near-miss", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "prism-words", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12115", + "score": 8.16292667388916, + "lexicalRank": 61 + }, + { + "id": "12116", + "score": 8.108729362487793, + "lexicalRank": 60 + }, + { + "id": "12350", + "score": 6.787620544433594, + "lexicalRank": 255 + }, + { + "id": "12280", + "score": 6.751938819885254, + "lexicalRank": 269 + }, + { + "id": "12310", + "score": 6.689011573791504, + "lexicalRank": 263 + }, + { + "id": "12300", + "score": 6.637208461761475, + "lexicalRank": 265 + }, + { + "id": "12355", + "score": 6.6052045822143555, + "lexicalRank": 254 + }, + { + "id": "12410", + "score": 6.5069379806518555, + "lexicalRank": 243 + }, + { + "id": "12240", + "score": 6.487910747528076, + "lexicalRank": 277 + }, + { + "id": "12200", + "score": 6.4574360847473145, + "lexicalRank": 285 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-typo", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12115", + "score": 7.70074987411499, + "lexicalRank": 61 + }, + { + "id": "12116", + "score": 7.129725933074951, + "lexicalRank": 60 + }, + { + "id": "12367", + "score": 5.932244777679443, + "lexicalRank": 9 + }, + { + "id": "12247", + "score": 5.629062175750732, + "lexicalRank": 33 + }, + { + "id": "12365", + "score": 5.586015701293945, + "lexicalRank": 252 + }, + { + "id": "12317", + "score": 5.5220842361450195, + "lexicalRank": 19 + }, + { + "id": "12177", + "score": 5.4242119789123535, + "lexicalRank": 47 + }, + { + "id": "12172", + "score": 5.330693244934082, + "lexicalRank": 48 + }, + { + "id": "12385", + "score": 5.287479877471924, + "lexicalRank": 248 + }, + { + "id": "12202", + "score": 5.262482166290283, + "lexicalRank": 42 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-beginnings", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12115", + "score": 8.649856567382812, + "lexicalRank": 61 + }, + { + "id": "12116", + "score": 7.992773532867432, + "lexicalRank": 60 + }, + { + "id": "12357", + "score": 6.6164374351501465, + "lexicalRank": 11 + }, + { + "id": "12402", + "score": 6.311557769775391, + "lexicalRank": 2 + }, + { + "id": "12367", + "score": 6.110666751861572, + "lexicalRank": 9 + }, + { + "id": "12272", + "score": 6.051764965057373, + "lexicalRank": 28 + }, + { + "id": "12317", + "score": 6.044123649597168, + "lexicalRank": 19 + }, + { + "id": "12117", + "score": 6.029598236083984, + "lexicalRank": 59 + }, + { + "id": "12142", + "score": 6.0267720222473145, + "lexicalRank": 54 + }, + { + "id": "12132", + "score": 5.9389519691467285, + "lexicalRank": 56 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-phrase", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "score": 2.7623348535297314 + }, + { + "id": "12407", + "score": 2.7623348535297314 + }, + { + "id": "12402", + "score": 2.7623348535297314 + }, + { + "id": "12397", + "score": 2.7623348535297314 + }, + { + "id": "12392", + "score": 2.7623348535297314 + }, + { + "id": "12387", + "score": 2.7623348535297314 + }, + { + "id": "12382", + "score": 2.7623348535297314 + }, + { + "id": "12377", + "score": 2.7623348535297314 + }, + { + "id": "12372", + "score": 2.7623348535297314 + }, + { + "id": "12367", + "score": 2.7623348535297314 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-from", + "relevant": { + "12115": 2 + }, + "hits": [ + { + "id": "12115", + "score": -3.4879183769226074, + "lexicalRank": 0 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + } + }, + { + "id": "prism-chat", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12345", + "score": -0.1605147272348404, + "lexicalRank": 96 + }, + { + "id": "12325", + "score": -0.25931647419929504, + "lexicalRank": 100 + }, + { + "id": "12340", + "score": -0.46640682220458984, + "lexicalRank": 97 + }, + { + "id": "12365", + "score": -0.5419776439666748, + "lexicalRank": 92 + }, + { + "id": "12355", + "score": -0.5792048573493958, + "lexicalRank": 94 + }, + { + "id": "12405", + "score": -0.6089483499526978, + "lexicalRank": 84 + }, + { + "id": "12380", + "score": -0.6718494296073914, + "lexicalRank": 89 + }, + { + "id": "12395", + "score": -0.6834031343460083, + "lexicalRank": 86 + }, + { + "id": "12415", + "score": -0.6878743767738342, + "lexicalRank": 82 + }, + { + "id": "12370", + "score": -0.705737829208374, + "lexicalRank": 91 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-date", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12116", + "score": -3.8352277278900146, + "lexicalRank": 0 + }, + { + "id": "12115", + "score": -5.049022197723389, + "lexicalRank": 1 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + } + }, + { + "id": "prism-and", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "score": 7.489838832938161 + }, + { + "id": "12407", + "score": 7.489838832938161 + }, + { + "id": "12402", + "score": 7.489838832938161 + }, + { + "id": "12397", + "score": 7.489838832938161 + }, + { + "id": "12392", + "score": 7.489838832938161 + }, + { + "id": "12387", + "score": 7.489838832938161 + }, + { + "id": "12382", + "score": 7.489838832938161 + }, + { + "id": "12377", + "score": 7.489838832938161 + }, + { + "id": "12372", + "score": 7.489838832938161 + }, + { + "id": "12367", + "score": 7.489838832938161 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-or", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [ + { + "id": "12412", + "score": 7.489838832938161 + }, + { + "id": "12407", + "score": 7.489838832938161 + }, + { + "id": "12402", + "score": 7.489838832938161 + }, + { + "id": "12397", + "score": 7.489838832938161 + }, + { + "id": "12392", + "score": 7.489838832938161 + }, + { + "id": "12387", + "score": 7.489838832938161 + }, + { + "id": "12382", + "score": 7.489838832938161 + }, + { + "id": "12377", + "score": 7.489838832938161 + }, + { + "id": "12372", + "score": 7.489838832938161 + }, + { + "id": "12367", + "score": 7.489838832938161 + } + ], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-paraphrase", + "relevant": { + "12115": 2, + "12116": 1 + }, + "hits": [], + "metrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + } + }, + { + "id": "prism-absent", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "prism-near-miss", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + } + ] + }, + "questions": { + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.6875, + "mrr": 1, + "ndcgAt10": 0.8659851083291633, + "noAnswerFalseHits": 0 + }, + "rows": [ + { + "id": "cobalt-question-decision", + "relevant": { + "70001": 2, + "70002": 1 + }, + "hits": [ + { + "id": "70001", + "score": 10.751850128173828, + "lexicalRank": 121 + }, + { + "id": "70002", + "score": 8.593709945678711, + "lexicalRank": 60 + }, + { + "id": "70140", + "score": 5.350794315338135, + "lexicalRank": 32 + }, + { + "id": "70055", + "score": 5.1161675453186035, + "lexicalRank": 49 + }, + { + "id": "70115", + "score": 5.016029357910156, + "lexicalRank": 37 + }, + { + "id": "70030", + "score": 4.977260112762451, + "lexicalRank": 54 + }, + { + "id": "70045", + "score": 4.861608028411865, + "lexicalRank": 51 + }, + { + "id": "70100", + "score": 4.83193302154541, + "lexicalRank": 40 + }, + { + "id": "70090", + "score": 4.808356761932373, + "lexicalRank": 42 + }, + { + "id": "70125", + "score": 4.754176616668701, + "lexicalRank": 35 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + } + }, + { + "id": "cobalt-question-question", + "relevant": { + "70001": 2, + "70002": 1 + }, + "hits": [ + { + "id": "70001", + "score": 7.23106575012207, + "lexicalRank": 121 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + } + }, + { + "id": "cobalt-question-no-answer", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "birch-question-decision", + "relevant": { + "70153": 2, + "70154": 1 + }, + "hits": [ + { + "id": "70153", + "score": 10.18606185913086, + "lexicalRank": 30 + }, + { + "id": "70154", + "score": 9.900846481323242, + "lexicalRank": 61 + }, + { + "id": "70197", + "score": 5.7486066818237305, + "lexicalRank": 52 + }, + { + "id": "70242", + "score": 5.041281700134277, + "lexicalRank": 43 + }, + { + "id": "70287", + "score": 4.884515285491943, + "lexicalRank": 34 + }, + { + "id": "70277", + "score": 4.856664657592773, + "lexicalRank": 36 + }, + { + "id": "70212", + "score": 4.822803020477295, + "lexicalRank": 49 + }, + { + "id": "70207", + "score": 4.788071632385254, + "lexicalRank": 50 + }, + { + "id": "70182", + "score": 4.78555154800415, + "lexicalRank": 55 + }, + { + "id": "70237", + "score": 4.770137786865234, + "lexicalRank": 44 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + } + }, + { + "id": "birch-question-question", + "relevant": { + "70153": 2, + "70154": 1 + }, + "hits": [ + { + "id": "70153", + "score": 10.70551586151123, + "lexicalRank": 30 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + } + }, + { + "id": "birch-question-no-answer", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "спектр-question-decision", + "relevant": { + "70305": 2, + "70306": 1 + }, + "hits": [ + { + "id": "70305", + "score": 9.470867156982422, + "lexicalRank": 31 + }, + { + "id": "70449", + "score": 4.701529502868652, + "lexicalRank": 33 + }, + { + "id": "70429", + "score": 4.574463844299316, + "lexicalRank": 37 + }, + { + "id": "70379", + "score": 4.497837543487549, + "lexicalRank": 47 + }, + { + "id": "70409", + "score": 4.488161563873291, + "lexicalRank": 41 + }, + { + "id": "70342", + "score": 4.469592094421387, + "lexicalRank": 114 + }, + { + "id": "70439", + "score": 4.46698522567749, + "lexicalRank": 35 + }, + { + "id": "70367", + "score": 4.462706089019775, + "lexicalRank": 109 + }, + { + "id": "70454", + "score": 4.4601030349731445, + "lexicalRank": 32 + }, + { + "id": "70414", + "score": 4.357937812805176, + "lexicalRank": 40 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + } + }, + { + "id": "спектр-question-question", + "relevant": { + "70305": 2, + "70306": 1 + }, + "hits": [ + { + "id": "70305", + "score": 7.549112319946289, + "lexicalRank": 31 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + } + }, + { + "id": "спектр-question-no-answer", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + }, + { + "id": "nova-question-decision", + "relevant": { + "70457": 2, + "70458": 1 + }, + "hits": [ + { + "id": "70458", + "score": 8.890884399414062, + "lexicalRank": 60 + }, + { + "id": "70457", + "score": 8.344308853149414, + "lexicalRank": 61 + }, + { + "id": "70581", + "score": 4.407783508300781, + "lexicalRank": 35 + }, + { + "id": "70526", + "score": 4.299479007720947, + "lexicalRank": 46 + }, + { + "id": "70556", + "score": 4.249410152435303, + "lexicalRank": 40 + }, + { + "id": "70566", + "score": 4.116221904754639, + "lexicalRank": 38 + }, + { + "id": "70501", + "score": 4.0909576416015625, + "lexicalRank": 51 + }, + { + "id": "70531", + "score": 3.9745075702667236, + "lexicalRank": 45 + }, + { + "id": "70516", + "score": 3.9253413677215576, + "lexicalRank": 48 + }, + { + "id": "70586", + "score": 3.8646199703216553, + "lexicalRank": 34 + } + ], + "metrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + } + }, + { + "id": "nova-question-question", + "relevant": { + "70457": 2, + "70458": 1 + }, + "hits": [ + { + "id": "70457", + "score": 10.7865629196167, + "lexicalRank": 61 + } + ], + "metrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + } + }, + { + "id": "nova-question-no-answer", + "relevant": {}, + "hits": [], + "metrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + } + } + ] + } + }, + "inference": { + "pairInferenceMs": 136790.54689099995, + "scoredPairs": 13464, + "averagePairMs": 10.159725704916811, + "loadMs": 1173.3451759999998, + "embeddedMessages": 0, + "uniqueEmbeddedQueries": 0, + "uniqueCrossEncoderQueries": 72, + "passageInferenceMs": 0, + "queryInferenceMs": 0, + "amortizedPerMessageMs": null, + "cache": "exact synthetic query-message score cache, empty at run start" + }, + "candidateRows": [ + { + "id": "aurora-words", + "category": "words", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-words", + "category": "words", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-words", + "category": "words", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-words", + "category": "words", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 50, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 100, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 300, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 500, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-and", + "category": "and", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-and", + "category": "and", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-and", + "category": "and", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-and", + "category": "and", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-or", + "category": "or", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-or", + "category": "or", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-or", + "category": "or", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-or", + "category": "or", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-words", + "category": "words", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-words", + "category": "words", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-words", + "category": "words", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-words", + "category": "words", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 50, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 100, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 300, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 500, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-and", + "category": "and", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-and", + "category": "and", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-and", + "category": "and", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-and", + "category": "and", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-or", + "category": "or", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-or", + "category": "or", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-or", + "category": "or", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-or", + "category": "or", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-words", + "category": "words", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-words", + "category": "words", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-words", + "category": "words", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-words", + "category": "words", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662" + ], + "candidateRecall": 0.5, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10605", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10605", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611" + ], + "candidateRecall": 1, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10605", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611" + ], + "candidateRecall": 1, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 50, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 100, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 300, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 500, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-and", + "category": "and", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-and", + "category": "and", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-and", + "category": "and", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-and", + "category": "and", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-or", + "category": "or", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-or", + "category": "or", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-or", + "category": "or", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-or", + "category": "or", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-words", + "category": "words", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-words", + "category": "words", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-words", + "category": "words", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-words", + "category": "words", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 50, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 100, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 300, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 500, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-and", + "category": "and", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-and", + "category": "and", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-and", + "category": "and", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-and", + "category": "and", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-or", + "category": "or", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-or", + "category": "or", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-or", + "category": "or", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-or", + "category": "or", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + } + ], + "keywordRows": [ + { + "id": "aurora-words", + "category": "words", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10001", + "score": 9.144113540649414, + "lexicalRank": 60 + }, + { + "id": "10002", + "score": 8.763883590698242, + "lexicalRank": 121 + }, + { + "id": "10046", + "score": 8.404559135437012, + "lexicalRank": 293 + }, + { + "id": "10211", + "score": 8.33627986907959, + "lexicalRank": 260 + }, + { + "id": "10086", + "score": 8.127921104431152, + "lexicalRank": 285 + }, + { + "id": "10186", + "score": 8.10696029663086, + "lexicalRank": 265 + }, + { + "id": "10236", + "score": 8.104728698730469, + "lexicalRank": 255 + }, + { + "id": "10191", + "score": 8.0871000289917, + "lexicalRank": 264 + }, + { + "id": "10146", + "score": 8.065347671508789, + "lexicalRank": 273 + }, + { + "id": "10116", + "score": 8.039112091064453, + "lexicalRank": 279 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-typo", + "category": "typo", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10002", + "score": 8.407031059265137, + "lexicalRank": 121 + }, + { + "id": "10001", + "score": 7.457723617553711, + "lexicalRank": 60 + }, + { + "id": "10090", + "score": 7.038037300109863, + "lexicalRank": 164 + }, + { + "id": "10218", + "score": 7.009069919586182, + "lexicalRank": 16 + }, + { + "id": "10055", + "score": 6.976167678833008, + "lexicalRank": 171 + }, + { + "id": "10233", + "score": 6.961915969848633, + "lexicalRank": 13 + }, + { + "id": "10153", + "score": 6.896630764007568, + "lexicalRank": 29 + }, + { + "id": "10247", + "score": 6.875993251800537, + "lexicalRank": 193 + }, + { + "id": "10223", + "score": 6.869973659515381, + "lexicalRank": 15 + }, + { + "id": "10015", + "score": 6.8225297927856445, + "lexicalRank": 179 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10002", + "score": 8.832911491394043, + "lexicalRank": 121 + }, + { + "id": "10001", + "score": 8.565581321716309, + "lexicalRank": 60 + }, + { + "id": "10055", + "score": 7.864835739135742, + "lexicalRank": 171 + }, + { + "id": "10205", + "score": 7.708717346191406, + "lexicalRank": 141 + }, + { + "id": "10175", + "score": 7.707800388336182, + "lexicalRank": 147 + }, + { + "id": "10155", + "score": 7.70137357711792, + "lexicalRank": 151 + }, + { + "id": "10150", + "score": 7.671929836273193, + "lexicalRank": 152 + }, + { + "id": "10065", + "score": 7.642721176147461, + "lexicalRank": 169 + }, + { + "id": "10210", + "score": 7.6175618171691895, + "lexicalRank": 140 + }, + { + "id": "10070", + "score": 7.612574100494385, + "lexicalRank": 168 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-phrase", + "category": "phrase", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 2.7623348535297314 + }, + { + "id": "10293", + "score": 2.7623348535297314 + }, + { + "id": "10288", + "score": 2.7623348535297314 + }, + { + "id": "10283", + "score": 2.7623348535297314 + }, + { + "id": "10278", + "score": 2.7623348535297314 + }, + { + "id": "10273", + "score": 2.7623348535297314 + }, + { + "id": "10268", + "score": 2.7623348535297314 + }, + { + "id": "10263", + "score": 2.7623348535297314 + }, + { + "id": "10258", + "score": 2.7623348535297314 + }, + { + "id": "10253", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "10298", + "score": 2.7623348535297314 + }, + { + "id": "10293", + "score": 2.7623348535297314 + }, + { + "id": "10288", + "score": 2.7623348535297314 + }, + { + "id": "10283", + "score": 2.7623348535297314 + }, + { + "id": "10278", + "score": 2.7623348535297314 + }, + { + "id": "10273", + "score": 2.7623348535297314 + }, + { + "id": "10268", + "score": 2.7623348535297314 + }, + { + "id": "10263", + "score": 2.7623348535297314 + }, + { + "id": "10258", + "score": 2.7623348535297314 + }, + { + "id": "10253", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "aurora-from", + "category": "from", + "relevant": { + "10001": 2 + }, + "lexical": [ + { + "id": "10001", + "score": 1 + } + ], + "semantic": [ + { + "id": "10001", + "score": -2.133422374725342, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-chat", + "category": "chat", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10206", + "score": 1.1320911645889282, + "lexicalRank": 101 + }, + { + "id": "10226", + "score": 1.127381682395935, + "lexicalRank": 97 + }, + { + "id": "10291", + "score": 1.0891575813293457, + "lexicalRank": 84 + }, + { + "id": "10221", + "score": 0.7220429182052612, + "lexicalRank": 98 + }, + { + "id": "10246", + "score": 0.6760910153388977, + "lexicalRank": 93 + }, + { + "id": "10296", + "score": 0.6750655770301819, + "lexicalRank": 83 + }, + { + "id": "10281", + "score": 0.6719631552696228, + "lexicalRank": 86 + }, + { + "id": "10231", + "score": 0.6679152250289917, + "lexicalRank": 96 + }, + { + "id": "10271", + "score": 0.654645562171936, + "lexicalRank": 88 + }, + { + "id": "10216", + "score": 0.5903054475784302, + "lexicalRank": 99 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-date", + "category": "date", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10001", + "score": 1 + }, + { + "id": "10002", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10001", + "score": -2.672081708908081, + "lexicalRank": 0 + }, + { + "id": "10002", + "score": -3.1790313720703125, + "lexicalRank": 1 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-and", + "category": "and", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "aurora-or", + "category": "or", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-words", + "category": "words", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10304", + "score": 8.97688102722168, + "lexicalRank": 121 + }, + { + "id": "10598", + "score": 7.926235198974609, + "lexicalRank": 243 + }, + { + "id": "10303", + "score": 7.766183376312256, + "lexicalRank": 60 + }, + { + "id": "10453", + "score": 7.723439693450928, + "lexicalRank": 272 + }, + { + "id": "10543", + "score": 7.701955795288086, + "lexicalRank": 254 + }, + { + "id": "10603", + "score": 7.637665748596191, + "lexicalRank": 242 + }, + { + "id": "10413", + "score": 7.637113094329834, + "lexicalRank": 280 + }, + { + "id": "10488", + "score": 7.61979866027832, + "lexicalRank": 265 + }, + { + "id": "10478", + "score": 7.59701681137085, + "lexicalRank": 267 + }, + { + "id": "10458", + "score": 7.580545902252197, + "lexicalRank": 271 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.6885288809404666, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-typo", + "category": "typo", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10304", + "score": 4.184720516204834, + "lexicalRank": 121 + }, + { + "id": "10303", + "score": 3.1427712440490723, + "lexicalRank": 60 + }, + { + "id": "10590", + "score": 2.2103934288024902, + "lexicalRank": 2 + }, + { + "id": "10547", + "score": 2.043490171432495, + "lexicalRank": 133 + }, + { + "id": "10347", + "score": 1.9820915460586548, + "lexicalRank": 173 + }, + { + "id": "10572", + "score": 1.9670615196228027, + "lexicalRank": 128 + }, + { + "id": "10372", + "score": 1.941902756690979, + "lexicalRank": 168 + }, + { + "id": "10467", + "score": 1.9225600957870483, + "lexicalRank": 149 + }, + { + "id": "10452", + "score": 1.9188569784164429, + "lexicalRank": 152 + }, + { + "id": "10482", + "score": 1.9111326932907104, + "lexicalRank": 146 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10304", + "score": 8.845378875732422, + "lexicalRank": 121 + }, + { + "id": "10303", + "score": 7.421807765960693, + "lexicalRank": 60 + }, + { + "id": "10548", + "score": 6.571941375732422, + "lexicalRank": 253 + }, + { + "id": "10503", + "score": 6.559657573699951, + "lexicalRank": 262 + }, + { + "id": "10433", + "score": 6.476851940155029, + "lexicalRank": 276 + }, + { + "id": "10378", + "score": 6.4482622146606445, + "lexicalRank": 287 + }, + { + "id": "10518", + "score": 6.384768962860107, + "lexicalRank": 259 + }, + { + "id": "10368", + "score": 6.379504680633545, + "lexicalRank": 289 + }, + { + "id": "10568", + "score": 6.364409923553467, + "lexicalRank": 249 + }, + { + "id": "10598", + "score": 6.302361488342285, + "lexicalRank": 243 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-phrase", + "category": "phrase", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 2.7623348535297314 + }, + { + "id": "10595", + "score": 2.7623348535297314 + }, + { + "id": "10590", + "score": 2.7623348535297314 + }, + { + "id": "10585", + "score": 2.7623348535297314 + }, + { + "id": "10580", + "score": 2.7623348535297314 + }, + { + "id": "10575", + "score": 2.7623348535297314 + }, + { + "id": "10570", + "score": 2.7623348535297314 + }, + { + "id": "10565", + "score": 2.7623348535297314 + }, + { + "id": "10560", + "score": 2.7623348535297314 + }, + { + "id": "10555", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "10600", + "score": 2.7623348535297314 + }, + { + "id": "10595", + "score": 2.7623348535297314 + }, + { + "id": "10590", + "score": 2.7623348535297314 + }, + { + "id": "10585", + "score": 2.7623348535297314 + }, + { + "id": "10580", + "score": 2.7623348535297314 + }, + { + "id": "10575", + "score": 2.7623348535297314 + }, + { + "id": "10570", + "score": 2.7623348535297314 + }, + { + "id": "10565", + "score": 2.7623348535297314 + }, + { + "id": "10560", + "score": 2.7623348535297314 + }, + { + "id": "10555", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "cedar-from", + "category": "from", + "relevant": { + "10303": 2 + }, + "lexical": [ + { + "id": "10303", + "score": 1 + } + ], + "semantic": [ + { + "id": "10303", + "score": -2.1878979206085205, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-chat", + "category": "chat", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10548", + "score": -0.05690937489271164, + "lexicalRank": 93 + }, + { + "id": "10593", + "score": -0.1841631978750229, + "lexicalRank": 84 + }, + { + "id": "10518", + "score": -0.2792830169200897, + "lexicalRank": 99 + }, + { + "id": "10598", + "score": -0.36233586072921753, + "lexicalRank": 83 + }, + { + "id": "10558", + "score": -0.36453109979629517, + "lexicalRank": 91 + }, + { + "id": "10513", + "score": -0.41426771879196167, + "lexicalRank": 100 + }, + { + "id": "10508", + "score": -0.42688512802124023, + "lexicalRank": 101 + }, + { + "id": "10304", + "score": -0.5078280568122864, + "lexicalRank": 41 + }, + { + "id": "10538", + "score": -0.5147302150726318, + "lexicalRank": 95 + }, + { + "id": "10573", + "score": -0.5198200345039368, + "lexicalRank": 88 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0.5, + "mrr": 0.125, + "ndcgAt10": 0.08688267143572001, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-date", + "category": "date", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10303", + "score": 1 + }, + { + "id": "10304", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10304", + "score": -1.9164950847625732, + "lexicalRank": 1 + }, + { + "id": "10303", + "score": -2.703486919403076, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-and", + "category": "and", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "cedar-or", + "category": "or", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-words", + "category": "words", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10605", + "score": 9.941312789916992, + "lexicalRank": 1 + }, + { + "id": "10807", + "score": 8.448219299316406, + "lexicalRank": 21 + }, + { + "id": "10822", + "score": 8.419659614562988, + "lexicalRank": 18 + }, + { + "id": "10632", + "score": 8.358668327331543, + "lexicalRank": 56 + }, + { + "id": "10767", + "score": 8.349440574645996, + "lexicalRank": 29 + }, + { + "id": "10742", + "score": 8.334734916687012, + "lexicalRank": 34 + }, + { + "id": "10847", + "score": 8.306585311889648, + "lexicalRank": 13 + }, + { + "id": "10852", + "score": 8.301280975341797, + "lexicalRank": 12 + }, + { + "id": "10837", + "score": 8.297823905944824, + "lexicalRank": 15 + }, + { + "id": "10797", + "score": 8.287442207336426, + "lexicalRank": 23 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-typo", + "category": "typo", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10801", + "score": 5.321130752563477, + "lexicalRank": 203 + }, + { + "id": "10866", + "score": 5.154948711395264, + "lexicalRank": 190 + }, + { + "id": "10721", + "score": 5.130404949188232, + "lexicalRank": 219 + }, + { + "id": "10726", + "score": 5.04014253616333, + "lexicalRank": 218 + }, + { + "id": "10711", + "score": 5.0147480964660645, + "lexicalRank": 221 + }, + { + "id": "10671", + "score": 4.996459484100342, + "lexicalRank": 229 + }, + { + "id": "10856", + "score": 4.986394882202148, + "lexicalRank": 192 + }, + { + "id": "10781", + "score": 4.971275329589844, + "lexicalRank": 207 + }, + { + "id": "10681", + "score": 4.969725608825684, + "lexicalRank": 227 + }, + { + "id": "10811", + "score": 4.945552349090576, + "lexicalRank": 201 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10605", + "score": 8.343690872192383, + "lexicalRank": 1 + }, + { + "id": "10632", + "score": 6.755748748779297, + "lexicalRank": 56 + }, + { + "id": "10727", + "score": 6.705746173858643, + "lexicalRank": 37 + }, + { + "id": "10607", + "score": 6.353328227996826, + "lexicalRank": 61 + }, + { + "id": "10747", + "score": 6.205280303955078, + "lexicalRank": 33 + }, + { + "id": "10692", + "score": 6.176116466522217, + "lexicalRank": 44 + }, + { + "id": "10667", + "score": 6.106525421142578, + "lexicalRank": 49 + }, + { + "id": "10657", + "score": 6.102047443389893, + "lexicalRank": 51 + }, + { + "id": "10736", + "score": 6.071948051452637, + "lexicalRank": 216 + }, + { + "id": "10622", + "score": 6.069979190826416, + "lexicalRank": 58 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-phrase", + "category": "phrase", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 3.126652600631594 + }, + { + "id": "10902", + "score": 2.7623348535297314 + }, + { + "id": "10897", + "score": 2.7623348535297314 + }, + { + "id": "10892", + "score": 2.7623348535297314 + }, + { + "id": "10887", + "score": 2.7623348535297314 + }, + { + "id": "10882", + "score": 2.7623348535297314 + }, + { + "id": "10877", + "score": 2.7623348535297314 + }, + { + "id": "10872", + "score": 2.7623348535297314 + }, + { + "id": "10867", + "score": 2.7623348535297314 + }, + { + "id": "10862", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "10606", + "score": 3.126652600631594 + }, + { + "id": "10902", + "score": 2.7623348535297314 + }, + { + "id": "10897", + "score": 2.7623348535297314 + }, + { + "id": "10892", + "score": 2.7623348535297314 + }, + { + "id": "10887", + "score": 2.7623348535297314 + }, + { + "id": "10882", + "score": 2.7623348535297314 + }, + { + "id": "10877", + "score": 2.7623348535297314 + }, + { + "id": "10872", + "score": 2.7623348535297314 + }, + { + "id": "10867", + "score": 2.7623348535297314 + }, + { + "id": "10862", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "север-from", + "category": "from", + "relevant": { + "10605": 2 + }, + "lexical": [ + { + "id": "10605", + "score": 1 + } + ], + "semantic": [ + { + "id": "10605", + "score": -1.4385977983474731, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-chat", + "category": "chat", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10605", + "score": 1 + }, + { + "id": "10606", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10810", + "score": -0.8035014867782593, + "lexicalRank": 101 + }, + { + "id": "10870", + "score": -0.8724918365478516, + "lexicalRank": 89 + }, + { + "id": "10820", + "score": -0.8758571147918701, + "lexicalRank": 99 + }, + { + "id": "10890", + "score": -0.9417440295219421, + "lexicalRank": 85 + }, + { + "id": "10900", + "score": -1.0117398500442505, + "lexicalRank": 83 + }, + { + "id": "10815", + "score": -1.0464911460876465, + "lexicalRank": 100 + }, + { + "id": "10825", + "score": -1.126318097114563, + "lexicalRank": 98 + }, + { + "id": "10855", + "score": -1.2063350677490234, + "lexicalRank": 92 + }, + { + "id": "10850", + "score": -1.2222322225570679, + "lexicalRank": 93 + }, + { + "id": "10880", + "score": -1.280145287513733, + "lexicalRank": 87 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-date", + "category": "date", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10605", + "score": 1 + }, + { + "id": "10606", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10605", + "score": 0.5484523177146912, + "lexicalRank": 0 + }, + { + "id": "10606", + "score": -3.554842233657837, + "lexicalRank": 1 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-and", + "category": "and", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "север-or", + "category": "or", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-words", + "category": "words", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10907", + "score": 9.74099349975586, + "lexicalRank": 61 + }, + { + "id": "10908", + "score": 9.090091705322266, + "lexicalRank": 60 + }, + { + "id": "11062", + "score": 6.556122779846191, + "lexicalRank": 271 + }, + { + "id": "11057", + "score": 6.3410139083862305, + "lexicalRank": 272 + }, + { + "id": "10942", + "score": 6.337686061859131, + "lexicalRank": 295 + }, + { + "id": "11052", + "score": 6.325874328613281, + "lexicalRank": 273 + }, + { + "id": "10922", + "score": 6.1378703117370605, + "lexicalRank": 299 + }, + { + "id": "11087", + "score": 6.117567539215088, + "lexicalRank": 266 + }, + { + "id": "11012", + "score": 6.073670864105225, + "lexicalRank": 281 + }, + { + "id": "10937", + "score": 6.069832801818848, + "lexicalRank": 296 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-typo", + "category": "typo", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10907", + "score": 9.766773223876953, + "lexicalRank": 61 + }, + { + "id": "10908", + "score": 8.03225326538086, + "lexicalRank": 60 + }, + { + "id": "11029", + "score": 6.531184673309326, + "lexicalRank": 35 + }, + { + "id": "11154", + "score": 6.201030254364014, + "lexicalRank": 10 + }, + { + "id": "10949", + "score": 6.130903244018555, + "lexicalRank": 51 + }, + { + "id": "11136", + "score": 6.114521026611328, + "lexicalRank": 136 + }, + { + "id": "10979", + "score": 5.984946250915527, + "lexicalRank": 45 + }, + { + "id": "10987", + "score": 5.975259304046631, + "lexicalRank": 286 + }, + { + "id": "11081", + "score": 5.9714884757995605, + "lexicalRank": 147 + }, + { + "id": "11199", + "score": 5.929468631744385, + "lexicalRank": 1 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10907", + "score": 9.180591583251953, + "lexicalRank": 61 + }, + { + "id": "10908", + "score": 8.467389106750488, + "lexicalRank": 60 + }, + { + "id": "10909", + "score": 6.508573532104492, + "lexicalRank": 59 + }, + { + "id": "11086", + "score": 6.209899425506592, + "lexicalRank": 146 + }, + { + "id": "11101", + "score": 6.07896614074707, + "lexicalRank": 143 + }, + { + "id": "10954", + "score": 6.045144081115723, + "lexicalRank": 50 + }, + { + "id": "11084", + "score": 6.017730236053467, + "lexicalRank": 24 + }, + { + "id": "10947", + "score": 6.009658336639404, + "lexicalRank": 294 + }, + { + "id": "11102", + "score": 5.957398891448975, + "lexicalRank": 263 + }, + { + "id": "11132", + "score": 5.930818557739258, + "lexicalRank": 257 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-phrase", + "category": "phrase", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 2.7623348535297314 + }, + { + "id": "11199", + "score": 2.7623348535297314 + }, + { + "id": "11194", + "score": 2.7623348535297314 + }, + { + "id": "11189", + "score": 2.7623348535297314 + }, + { + "id": "11184", + "score": 2.7623348535297314 + }, + { + "id": "11179", + "score": 2.7623348535297314 + }, + { + "id": "11174", + "score": 2.7623348535297314 + }, + { + "id": "11169", + "score": 2.7623348535297314 + }, + { + "id": "11164", + "score": 2.7623348535297314 + }, + { + "id": "11159", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "11204", + "score": 2.7623348535297314 + }, + { + "id": "11199", + "score": 2.7623348535297314 + }, + { + "id": "11194", + "score": 2.7623348535297314 + }, + { + "id": "11189", + "score": 2.7623348535297314 + }, + { + "id": "11184", + "score": 2.7623348535297314 + }, + { + "id": "11179", + "score": 2.7623348535297314 + }, + { + "id": "11174", + "score": 2.7623348535297314 + }, + { + "id": "11169", + "score": 2.7623348535297314 + }, + { + "id": "11164", + "score": 2.7623348535297314 + }, + { + "id": "11159", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "lumen-from", + "category": "from", + "relevant": { + "10907": 2 + }, + "lexical": [ + { + "id": "10907", + "score": 1 + } + ], + "semantic": [ + { + "id": "10907", + "score": -1.6173491477966309, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-chat", + "category": "chat", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "11177", + "score": -1.4219605922698975, + "lexicalRank": 88 + }, + { + "id": "11122", + "score": -1.5253188610076904, + "lexicalRank": 99 + }, + { + "id": "11142", + "score": -1.5526238679885864, + "lexicalRank": 95 + }, + { + "id": "11172", + "score": -1.5592488050460815, + "lexicalRank": 89 + }, + { + "id": "11137", + "score": -1.5898562669754028, + "lexicalRank": 96 + }, + { + "id": "11132", + "score": -1.6688312292099, + "lexicalRank": 97 + }, + { + "id": "11192", + "score": -1.6710636615753174, + "lexicalRank": 85 + }, + { + "id": "11152", + "score": -1.6879723072052002, + "lexicalRank": 93 + }, + { + "id": "11162", + "score": -1.6941324472427368, + "lexicalRank": 91 + }, + { + "id": "11147", + "score": -1.7030967473983765, + "lexicalRank": 94 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-date", + "category": "date", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "10908", + "score": 1 + }, + { + "id": "10907", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10908", + "score": -3.8795433044433594, + "lexicalRank": 0 + }, + { + "id": "10907", + "score": -4.088683605194092, + "lexicalRank": 1 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-and", + "category": "and", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "lumen-or", + "category": "or", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + } + ], + "questionRows": [ + { + "id": "aurora-question-decision", + "category": "decision", + "relevant": { + "10001": 2, + "10002": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10001", + "score": 8.382225036621094, + "lexicalRank": 60 + }, + { + "id": "10302", + "score": 7.452626705169678, + "lexicalRank": 182 + }, + { + "id": "10022", + "score": 7.383761882781982, + "lexicalRank": 238 + }, + { + "id": "10122", + "score": 7.347390174865723, + "lexicalRank": 218 + }, + { + "id": "10297", + "score": 7.336249351501465, + "lexicalRank": 183 + }, + { + "id": "10172", + "score": 7.281764030456543, + "lexicalRank": 208 + }, + { + "id": "10167", + "score": 7.272385597229004, + "lexicalRank": 209 + }, + { + "id": "10067", + "score": 7.264976978302002, + "lexicalRank": 229 + }, + { + "id": "10097", + "score": 7.242596626281738, + "lexicalRank": 223 + }, + { + "id": "10192", + "score": 7.228582382202148, + "lexicalRank": 204 + } + ], + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "aurora-question-question", + "category": "question", + "relevant": { + "10001": 2, + "10002": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10001", + "score": 5.120489597320557, + "lexicalRank": 60 + }, + { + "id": "10002", + "score": 0.6442391872406006, + "lexicalRank": 121 + }, + { + "id": "10046", + "score": 0.20066462457180023, + "lexicalRank": 293 + }, + { + "id": "10221", + "score": 0.1334114372730255, + "lexicalRank": 258 + }, + { + "id": "10146", + "score": -0.015001742169260979, + "lexicalRank": 273 + }, + { + "id": "10051", + "score": -0.03679070249199867, + "lexicalRank": 292 + }, + { + "id": "10201", + "score": -0.08005551248788834, + "lexicalRank": 262 + }, + { + "id": "10151", + "score": -0.10989139974117279, + "lexicalRank": 272 + }, + { + "id": "10136", + "score": -0.13825947046279907, + "lexicalRank": 275 + }, + { + "id": "10111", + "score": -0.1633528172969818, + "lexicalRank": 280 + } + ], + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "aurora-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10002", + "score": -0.025841891765594482, + "lexicalRank": 121 + }, + { + "id": "10216", + "score": -3.283686399459839, + "lexicalRank": 259 + }, + { + "id": "10141", + "score": -3.294750928878784, + "lexicalRank": 274 + }, + { + "id": "10176", + "score": -3.396474838256836, + "lexicalRank": 267 + }, + { + "id": "10221", + "score": -3.5081534385681152, + "lexicalRank": 258 + }, + { + "id": "10226", + "score": -3.5380752086639404, + "lexicalRank": 257 + }, + { + "id": "10111", + "score": -3.5499141216278076, + "lexicalRank": 280 + }, + { + "id": "10201", + "score": -3.5694005489349365, + "lexicalRank": 262 + }, + { + "id": "10031", + "score": -3.581321954727173, + "lexicalRank": 296 + }, + { + "id": "10086", + "score": -3.582766532897949, + "lexicalRank": 285 + } + ], + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "cedar-question-decision", + "category": "decision", + "relevant": { + "10303": 2, + "10304": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10304", + "score": 5.710914134979248, + "lexicalRank": 121 + }, + { + "id": "10303", + "score": 5.477982044219971, + "lexicalRank": 60 + }, + { + "id": "10524", + "score": 5.470695495605469, + "lexicalRank": 198 + }, + { + "id": "10539", + "score": 5.3812785148620605, + "lexicalRank": 195 + }, + { + "id": "10309", + "score": 5.204921245574951, + "lexicalRank": 241 + }, + { + "id": "10464", + "score": 5.174740314483643, + "lexicalRank": 210 + }, + { + "id": "10454", + "score": 5.0796356201171875, + "lexicalRank": 212 + }, + { + "id": "10494", + "score": 5.079585552215576, + "lexicalRank": 204 + }, + { + "id": "10509", + "score": 5.048130989074707, + "lexicalRank": 201 + }, + { + "id": "10559", + "score": 5.0241827964782715, + "lexicalRank": 191 + } + ], + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "cedar-question-question", + "category": "question", + "relevant": { + "10303": 2, + "10304": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10303", + "score": 10.953361511230469, + "lexicalRank": 60 + }, + { + "id": "10365", + "score": -4.5399861335754395, + "lexicalRank": 47 + }, + { + "id": "10500", + "score": -4.553804874420166, + "lexicalRank": 20 + }, + { + "id": "10450", + "score": -4.634690284729004, + "lexicalRank": 30 + }, + { + "id": "10355", + "score": -4.663148403167725, + "lexicalRank": 49 + }, + { + "id": "10395", + "score": -4.685377597808838, + "lexicalRank": 41 + }, + { + "id": "10340", + "score": -4.763490200042725, + "lexicalRank": 52 + }, + { + "id": "10304", + "score": -4.767633438110352, + "lexicalRank": 121 + }, + { + "id": "10590", + "score": -4.7967634201049805, + "lexicalRank": 2 + }, + { + "id": "10465", + "score": -4.799666881561279, + "lexicalRank": 27 + } + ], + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "cedar-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10304", + "score": 0.1637496054172516, + "lexicalRank": 121 + }, + { + "id": "10468", + "score": -3.1079888343811035, + "lexicalRank": 269 + }, + { + "id": "10413", + "score": -3.2417991161346436, + "lexicalRank": 280 + }, + { + "id": "10348", + "score": -3.2629144191741943, + "lexicalRank": 293 + }, + { + "id": "10593", + "score": -3.302618980407715, + "lexicalRank": 244 + }, + { + "id": "10433", + "score": -3.3093817234039307, + "lexicalRank": 276 + }, + { + "id": "10583", + "score": -3.3447654247283936, + "lexicalRank": 246 + }, + { + "id": "10499", + "score": -3.363835573196411, + "lexicalRank": 203 + }, + { + "id": "10509", + "score": -3.382288694381714, + "lexicalRank": 201 + }, + { + "id": "10488", + "score": -3.3837873935699463, + "lexicalRank": 265 + } + ], + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "север-question-decision", + "category": "decision", + "relevant": { + "10605": 2, + "10606": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10721", + "score": 4.2955641746521, + "lexicalRank": 219 + }, + { + "id": "10796", + "score": 4.140763759613037, + "lexicalRank": 204 + }, + { + "id": "10776", + "score": 4.033090114593506, + "lexicalRank": 208 + }, + { + "id": "10716", + "score": 3.98829984664917, + "lexicalRank": 220 + }, + { + "id": "10841", + "score": 3.8905200958251953, + "lexicalRank": 195 + }, + { + "id": "10696", + "score": 3.8901102542877197, + "lexicalRank": 224 + }, + { + "id": "10866", + "score": 3.8856360912323, + "lexicalRank": 190 + }, + { + "id": "10741", + "score": 3.87813663482666, + "lexicalRank": 215 + }, + { + "id": "10711", + "score": 3.8500914573669434, + "lexicalRank": 221 + }, + { + "id": "10671", + "score": 3.797639846801758, + "lexicalRank": 229 + } + ], + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "север-question-question", + "category": "question", + "relevant": { + "10605": 2, + "10606": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10605", + "score": 6.176127910614014, + "lexicalRank": 1 + }, + { + "id": "10700", + "score": 0.24041175842285156, + "lexicalRank": 283 + }, + { + "id": "10725", + "score": 0.09185468405485153, + "lexicalRank": 278 + }, + { + "id": "10740", + "score": 0.08189026266336441, + "lexicalRank": 275 + }, + { + "id": "10617", + "score": 0.055311575531959534, + "lexicalRank": 59 + }, + { + "id": "10680", + "score": 0.01837293803691864, + "lexicalRank": 287 + }, + { + "id": "10815", + "score": -0.005716301035135984, + "lexicalRank": 260 + }, + { + "id": "10842", + "score": -0.04512234404683113, + "lexicalRank": 14 + }, + { + "id": "10750", + "score": -0.05214595049619675, + "lexicalRank": 273 + }, + { + "id": "10675", + "score": -0.061676062643527985, + "lexicalRank": 288 + } + ], + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "север-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10606", + "score": 1.492608666419983, + "lexicalRank": 0 + }, + { + "id": "10605", + "score": -0.41894200444221497, + "lexicalRank": 1 + }, + { + "id": "10882", + "score": -0.9125255346298218, + "lexicalRank": 6 + }, + { + "id": "10757", + "score": -1.0422241687774658, + "lexicalRank": 31 + }, + { + "id": "10677", + "score": -1.0783100128173828, + "lexicalRank": 47 + }, + { + "id": "10852", + "score": -1.0828702449798584, + "lexicalRank": 12 + }, + { + "id": "10747", + "score": -1.0937329530715942, + "lexicalRank": 33 + }, + { + "id": "10687", + "score": -1.133028268814087, + "lexicalRank": 45 + }, + { + "id": "10637", + "score": -1.160571813583374, + "lexicalRank": 55 + }, + { + "id": "10832", + "score": -1.1662486791610718, + "lexicalRank": 16 + } + ], + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "lumen-question-decision", + "category": "decision", + "relevant": { + "10907": 2, + "10908": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10907", + "score": 10.767654418945312, + "lexicalRank": 61 + }, + { + "id": "10908", + "score": 6.997429370880127, + "lexicalRank": 60 + }, + { + "id": "11094", + "score": 2.228569269180298, + "lexicalRank": 22 + }, + { + "id": "11144", + "score": 2.126849412918091, + "lexicalRank": 12 + }, + { + "id": "11138", + "score": 2.1206576824188232, + "lexicalRank": 196 + }, + { + "id": "11008", + "score": 2.09920597076416, + "lexicalRank": 222 + }, + { + "id": "10978", + "score": 2.0736210346221924, + "lexicalRank": 228 + }, + { + "id": "10948", + "score": 2.0597970485687256, + "lexicalRank": 234 + }, + { + "id": "11118", + "score": 2.0094153881073, + "lexicalRank": 200 + }, + { + "id": "10969", + "score": 1.9855324029922485, + "lexicalRank": 47 + } + ], + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "lumen-question-question", + "category": "question", + "relevant": { + "10907": 2, + "10908": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10907", + "score": 10.034287452697754, + "lexicalRank": 61 + }, + { + "id": "11132", + "score": -0.9715840816497803, + "lexicalRank": 257 + }, + { + "id": "11012", + "score": -1.0529500246047974, + "lexicalRank": 281 + }, + { + "id": "11072", + "score": -1.053822636604309, + "lexicalRank": 269 + }, + { + "id": "11152", + "score": -1.0734130144119263, + "lexicalRank": 253 + }, + { + "id": "11077", + "score": -1.095531702041626, + "lexicalRank": 268 + }, + { + "id": "11157", + "score": -1.1130152940750122, + "lexicalRank": 252 + }, + { + "id": "11107", + "score": -1.1273714303970337, + "lexicalRank": 262 + }, + { + "id": "11127", + "score": -1.1431617736816406, + "lexicalRank": 258 + }, + { + "id": "11042", + "score": -1.1596165895462036, + "lexicalRank": 275 + } + ], + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "lumen-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10908", + "score": 0.5671241879463196, + "lexicalRank": 60 + }, + { + "id": "11192", + "score": -2.253211259841919, + "lexicalRank": 245 + }, + { + "id": "11107", + "score": -2.3072736263275146, + "lexicalRank": 262 + }, + { + "id": "10907", + "score": -2.4978764057159424, + "lexicalRank": 61 + }, + { + "id": "11092", + "score": -2.517425537109375, + "lexicalRank": 265 + }, + { + "id": "10997", + "score": -2.5706686973571777, + "lexicalRank": 284 + }, + { + "id": "10967", + "score": -2.5796711444854736, + "lexicalRank": 290 + }, + { + "id": "11142", + "score": -2.6291210651397705, + "lexicalRank": 255 + }, + { + "id": "11022", + "score": -2.6314783096313477, + "lexicalRank": 279 + }, + { + "id": "11137", + "score": -2.6895358562469482, + "lexicalRank": 256 + } + ], + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ] + } + ] +} diff --git a/bench/message-search-quality/crossencoder.ts b/bench/message-search-quality/crossencoder.ts new file mode 100644 index 000000000..3d2201fdb --- /dev/null +++ b/bench/message-search-quality/crossencoder.ts @@ -0,0 +1,66 @@ +import assert from "node:assert/strict" +import { createHash } from "node:crypto" +import { readFileSync } from "node:fs" +import { join } from "node:path" + +export const openCrossEncoder = async (directory: string) => { + const manifest = JSON.parse(readFileSync(new URL("./reranker.json", import.meta.url), "utf8")) as { + repository: string + revision: string + licence: string + maxTokens: number + files: { name: string; sha256: string; url: string }[] + } + for (const file of manifest.files) + assert.equal( + createHash("sha256") + .update(readFileSync(join(directory, file.name))) + .digest("hex"), + file.sha256, + `pinned reranker ${file.name}`, + ) + const config = JSON.parse(readFileSync(join(directory, "config.json"), "utf8")) as { + sbert_ce_default_activation_function: string + id2label: Record + } + assert.equal(config.sbert_ce_default_activation_function, "torch.nn.modules.linear.Identity") + assert.equal(Object.keys(config.id2label).length, 1) + const [{ Tokenizer }, ort] = await Promise.all([ + import("@huggingface/tokenizers"), + import("@leemour/cli-messaging-onnx"), + ]) + const tokenizer = new Tokenizer( + JSON.parse(readFileSync(join(directory, "tokenizer.json"), "utf8")), + JSON.parse(readFileSync(join(directory, "tokenizer_config.json"), "utf8")), + ) + ort.env.wasm.numThreads = 8 + const model = manifest.files.find((file) => file.name.endsWith(".onnx")) + assert.ok(model) + const session = await ort.InferenceSession.create(readFileSync(join(directory, model.name))) + assert.deepEqual([...session.inputNames].sort(), ["attention_mask", "input_ids"]) + assert.deepEqual(session.outputNames, ["logits"]) + return { + model: { + id: manifest.repository, + revision: manifest.revision, + licence: manifest.licence, + files: manifest.files.map(({ name, sha256 }) => ({ name, sha256 })), + threads: 8, + }, + score: async (query: string, passage: string) => { + const ids = tokenizer.encode(query, { text_pair: passage }).ids + assert.ok(ids.length <= manifest.maxTokens, "benchmark pair must fit without truncation") + const result = await session.run({ + input_ids: new ort.Tensor("int64", BigInt64Array.from(ids, BigInt), [1, ids.length]), + attention_mask: new ort.Tensor("int64", new BigInt64Array(ids.length).fill(1n), [1, ids.length]), + }) + const logits = result.logits + assert.ok(logits) + assert.equal(logits.data.length, 1) + const score = Number(logits.data[0]) + assert.ok(Number.isFinite(score)) + return score + }, + close: () => session.release(), + } +} diff --git a/bench/message-search-quality/generate-questions.py b/bench/message-search-quality/generate-questions.py new file mode 100644 index 000000000..3d8be606d --- /dev/null +++ b/bench/message-search-quality/generate-questions.py @@ -0,0 +1,50 @@ +import json +from pathlib import Path + +root = Path(__file__).parent +original = json.loads((root / 'corpus.json').read_text()) +queries, chats, messages = [], [], [] + +# Questions state the task, never the labelled answer. Original keyword queries remain unchanged. +questions = [ + ('aurora', 'aurora rollout rollback', 'What rollback decision was agreed for the Aurora rollout?', 'What should we do to roll back the Aurora rollout?'), + ('cedar', 'cedar backup retention', 'What retention policy was agreed for Cedar backups?', 'How long should Cedar backup snapshots be kept?'), + ('север', 'север сертификат обновление', 'Какое решение принято по обновлению сертификата Север?', 'Что известно о новом сертификате Север и сроке его действия?'), + ('lumen', 'lumen invoice refund', 'What refund was approved for the Lumen invoice?', 'What should be returned to the payer of the Lumen invoice?'), +] +for family, anchors, decision, question in questions: + relevant = next(q['relevant'] for q in original['queries'] if q['id'] == f'{family}-words') + for category, text in [('decision', decision), ('question', question)]: + queries.append(dict(id=f'{family}-question-{category}', split='dev', family=family, category=category, retrieval=anchors, text=text, relevant=relevant)) + text = f'Which independent auditor signed off on {anchors}?' if family != 'север' else 'Какой независимый аудитор подписал обновление сертификата Север?' + queries.append(dict(id=f'{family}-question-no-answer', split='dev', family=family, category='no-answer', retrieval=anchors, text=text, relevant={})) + +fresh = [ + ('cobalt', 'cobalt failover routing', 'Cobalt failover routing approved: route traffic to the west pool and hold the east pool offline.', 'Cobalt failover routing owner is Lea; the approved change is documented.', 'What failover routing was approved for Cobalt?', 'Where should Cobalt traffic go after failover?'), + ('birch', 'birch export encryption', 'Birch export encryption policy: encrypt every export with a per-recipient key before sending it.', 'Birch export encryption owner is Arun; the agreed policy is documented.', 'What export encryption policy was agreed for Birch?', 'How should Birch exports be encrypted before they are sent?'), + ('спектр', 'спектр очередь лимит', 'Спектр очередь лимит согласован: принимать не более 120 задач, остальные отправлять в резервную очередь.', 'Спектр очередь лимит подтвержден, владелец Даша.', 'Какой лимит очереди согласован для Спектр?', 'Сколько задач принимает очередь Спектр и куда отправлять остальные?'), + ('nova', 'nova access expiry', 'Nova access expiry policy: expire temporary access after 48 hours and require a new approval to extend it.', 'Nova access expiry owner is Bo; the agreed policy is documented.', 'What access expiry policy was agreed for Nova?', 'When does Nova temporary access expire and how can it be extended?'), +] +mid = 70000 +for i, (family, anchors, answer, support, decision, question) in enumerate(fresh, 201): + cid = str(i) + chats.append(dict(id=cid, title=f'Synthetic {family} decisions')) + relevant = {} + for grade, text in [(2, answer), (1, support)]: + mid += 1 + messages.append(dict(id=str(mid), chatId=cid, senderId='800', timestamp='2026-10-05T12:00:00.000Z', text=text)) + relevant[str(mid)] = grade + for j in range(150): + text = [f'{anchors}: put this item on the agenda. No agreement was reached.', + f'{anchors}: rejected suggestion copied from a draft, awaiting a decision.', + f'{anchors}: empty form awaiting the actual policy.', + f'Checking whether the team discussed {anchors}. This is a question, not a confirmed decision.', + f'{family} keyword list: {anchors}; no instruction is recorded.'][j % 5] + mid += 1 + messages.append(dict(id=str(mid), chatId=cid, senderId='801', timestamp='2026-10-06T12:00:00.000Z', text=f'{text} Item {j}.')) + for category, text in [('decision', decision), ('question', question)]: + queries.append(dict(id=f'{family}-question-{category}', split='test', family=family, category=category, retrieval=anchors, text=text, relevant=dict(relevant))) + text = f'Which independent auditor signed off on {anchors}?' if family != 'спектр' else 'Какой независимый аудитор подписал лимит очереди Спектр?' + queries.append(dict(id=f'{family}-question-no-answer', split='test', family=family, category='no-answer', retrieval=anchors, text=text, relevant={})) + +(root / 'questions.json').write_text(json.dumps(dict(chats=chats, messages=messages, queries=queries), ensure_ascii=False, indent=2) + '\n') diff --git a/bench/message-search-quality/prepare-reranker.py b/bench/message-search-quality/prepare-reranker.py new file mode 100644 index 000000000..b20643a69 --- /dev/null +++ b/bench/message-search-quality/prepare-reranker.py @@ -0,0 +1,25 @@ +"""Explicit model preparation, kept separate from the offline benchmark.""" +import hashlib +import json +import pathlib +import sys +import urllib.request + +root = pathlib.Path(sys.argv[1]) +manifest = json.loads(pathlib.Path(__file__).with_name('reranker.json').read_text()) +root.mkdir(parents=True, exist_ok=True) +for file in manifest['files']: + target = root / file['name'] + target.parent.mkdir(parents=True, exist_ok=True) + if not target.exists(): + staged = target.with_name(target.name + '.partial') + print('Preparing pinned model file:', file['name'], flush=True) + with urllib.request.urlopen(file['url']) as response, staged.open('wb') as output: + while block := response.read(1024 * 1024): + output.write(block) + actual = hashlib.sha256(staged.read_bytes()).hexdigest() + if actual != file['sha256']: + raise ValueError(f"Hash mismatch for {file['name']}; partial file retained for inspection") + staged.rename(target) + assert hashlib.sha256(target.read_bytes()).hexdigest() == file['sha256'], file['name'] +print('Verified local model directory:', root) diff --git a/bench/message-search-quality/questions.json b/bench/message-search-quality/questions.json new file mode 100644 index 000000000..c075065fa --- /dev/null +++ b/bench/message-search-quality/questions.json @@ -0,0 +1,4544 @@ +{ + "chats": [ + { + "id": "201", + "title": "Synthetic cobalt decisions" + }, + { + "id": "202", + "title": "Synthetic birch decisions" + }, + { + "id": "203", + "title": "Synthetic спектр decisions" + }, + { + "id": "204", + "title": "Synthetic nova decisions" + } + ], + "messages": [ + { + "id": "70001", + "chatId": "201", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Cobalt failover routing approved: route traffic to the west pool and hold the east pool offline." + }, + { + "id": "70002", + "chatId": "201", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Cobalt failover routing owner is Lea; the approved change is documented." + }, + { + "id": "70003", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 0." + }, + { + "id": "70004", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 1." + }, + { + "id": "70005", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 2." + }, + { + "id": "70006", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 3." + }, + { + "id": "70007", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 4." + }, + { + "id": "70008", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 5." + }, + { + "id": "70009", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 6." + }, + { + "id": "70010", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 7." + }, + { + "id": "70011", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 8." + }, + { + "id": "70012", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 9." + }, + { + "id": "70013", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 10." + }, + { + "id": "70014", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 11." + }, + { + "id": "70015", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 12." + }, + { + "id": "70016", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 13." + }, + { + "id": "70017", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 14." + }, + { + "id": "70018", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 15." + }, + { + "id": "70019", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 16." + }, + { + "id": "70020", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 17." + }, + { + "id": "70021", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 18." + }, + { + "id": "70022", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 19." + }, + { + "id": "70023", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 20." + }, + { + "id": "70024", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 21." + }, + { + "id": "70025", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 22." + }, + { + "id": "70026", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 23." + }, + { + "id": "70027", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 24." + }, + { + "id": "70028", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 25." + }, + { + "id": "70029", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 26." + }, + { + "id": "70030", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 27." + }, + { + "id": "70031", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 28." + }, + { + "id": "70032", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 29." + }, + { + "id": "70033", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 30." + }, + { + "id": "70034", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 31." + }, + { + "id": "70035", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 32." + }, + { + "id": "70036", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 33." + }, + { + "id": "70037", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 34." + }, + { + "id": "70038", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 35." + }, + { + "id": "70039", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 36." + }, + { + "id": "70040", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 37." + }, + { + "id": "70041", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 38." + }, + { + "id": "70042", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 39." + }, + { + "id": "70043", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 40." + }, + { + "id": "70044", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 41." + }, + { + "id": "70045", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 42." + }, + { + "id": "70046", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 43." + }, + { + "id": "70047", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 44." + }, + { + "id": "70048", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 45." + }, + { + "id": "70049", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 46." + }, + { + "id": "70050", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 47." + }, + { + "id": "70051", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 48." + }, + { + "id": "70052", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 49." + }, + { + "id": "70053", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 50." + }, + { + "id": "70054", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 51." + }, + { + "id": "70055", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 52." + }, + { + "id": "70056", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 53." + }, + { + "id": "70057", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 54." + }, + { + "id": "70058", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 55." + }, + { + "id": "70059", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 56." + }, + { + "id": "70060", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 57." + }, + { + "id": "70061", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 58." + }, + { + "id": "70062", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 59." + }, + { + "id": "70063", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 60." + }, + { + "id": "70064", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 61." + }, + { + "id": "70065", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 62." + }, + { + "id": "70066", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 63." + }, + { + "id": "70067", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 64." + }, + { + "id": "70068", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 65." + }, + { + "id": "70069", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 66." + }, + { + "id": "70070", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 67." + }, + { + "id": "70071", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 68." + }, + { + "id": "70072", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 69." + }, + { + "id": "70073", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 70." + }, + { + "id": "70074", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 71." + }, + { + "id": "70075", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 72." + }, + { + "id": "70076", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 73." + }, + { + "id": "70077", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 74." + }, + { + "id": "70078", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 75." + }, + { + "id": "70079", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 76." + }, + { + "id": "70080", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 77." + }, + { + "id": "70081", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 78." + }, + { + "id": "70082", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 79." + }, + { + "id": "70083", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 80." + }, + { + "id": "70084", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 81." + }, + { + "id": "70085", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 82." + }, + { + "id": "70086", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 83." + }, + { + "id": "70087", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 84." + }, + { + "id": "70088", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 85." + }, + { + "id": "70089", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 86." + }, + { + "id": "70090", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 87." + }, + { + "id": "70091", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 88." + }, + { + "id": "70092", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 89." + }, + { + "id": "70093", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 90." + }, + { + "id": "70094", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 91." + }, + { + "id": "70095", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 92." + }, + { + "id": "70096", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 93." + }, + { + "id": "70097", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 94." + }, + { + "id": "70098", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 95." + }, + { + "id": "70099", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 96." + }, + { + "id": "70100", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 97." + }, + { + "id": "70101", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 98." + }, + { + "id": "70102", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 99." + }, + { + "id": "70103", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 100." + }, + { + "id": "70104", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 101." + }, + { + "id": "70105", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 102." + }, + { + "id": "70106", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 103." + }, + { + "id": "70107", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 104." + }, + { + "id": "70108", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 105." + }, + { + "id": "70109", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 106." + }, + { + "id": "70110", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 107." + }, + { + "id": "70111", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 108." + }, + { + "id": "70112", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 109." + }, + { + "id": "70113", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 110." + }, + { + "id": "70114", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 111." + }, + { + "id": "70115", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 112." + }, + { + "id": "70116", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 113." + }, + { + "id": "70117", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 114." + }, + { + "id": "70118", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 115." + }, + { + "id": "70119", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 116." + }, + { + "id": "70120", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 117." + }, + { + "id": "70121", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 118." + }, + { + "id": "70122", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 119." + }, + { + "id": "70123", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 120." + }, + { + "id": "70124", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 121." + }, + { + "id": "70125", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 122." + }, + { + "id": "70126", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 123." + }, + { + "id": "70127", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 124." + }, + { + "id": "70128", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 125." + }, + { + "id": "70129", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 126." + }, + { + "id": "70130", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 127." + }, + { + "id": "70131", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 128." + }, + { + "id": "70132", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 129." + }, + { + "id": "70133", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 130." + }, + { + "id": "70134", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 131." + }, + { + "id": "70135", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 132." + }, + { + "id": "70136", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 133." + }, + { + "id": "70137", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 134." + }, + { + "id": "70138", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 135." + }, + { + "id": "70139", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 136." + }, + { + "id": "70140", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 137." + }, + { + "id": "70141", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 138." + }, + { + "id": "70142", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 139." + }, + { + "id": "70143", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 140." + }, + { + "id": "70144", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 141." + }, + { + "id": "70145", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 142." + }, + { + "id": "70146", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 143." + }, + { + "id": "70147", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 144." + }, + { + "id": "70148", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 145." + }, + { + "id": "70149", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 146." + }, + { + "id": "70150", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt failover routing: empty form awaiting the actual policy. Item 147." + }, + { + "id": "70151", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 148." + }, + { + "id": "70152", + "chatId": "201", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 149." + }, + { + "id": "70153", + "chatId": "202", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Birch export encryption policy: encrypt every export with a per-recipient key before sending it." + }, + { + "id": "70154", + "chatId": "202", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Birch export encryption owner is Arun; the agreed policy is documented." + }, + { + "id": "70155", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 0." + }, + { + "id": "70156", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 1." + }, + { + "id": "70157", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 2." + }, + { + "id": "70158", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 3." + }, + { + "id": "70159", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 4." + }, + { + "id": "70160", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 5." + }, + { + "id": "70161", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 6." + }, + { + "id": "70162", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 7." + }, + { + "id": "70163", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 8." + }, + { + "id": "70164", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 9." + }, + { + "id": "70165", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 10." + }, + { + "id": "70166", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 11." + }, + { + "id": "70167", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 12." + }, + { + "id": "70168", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 13." + }, + { + "id": "70169", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 14." + }, + { + "id": "70170", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 15." + }, + { + "id": "70171", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 16." + }, + { + "id": "70172", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 17." + }, + { + "id": "70173", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 18." + }, + { + "id": "70174", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 19." + }, + { + "id": "70175", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 20." + }, + { + "id": "70176", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 21." + }, + { + "id": "70177", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 22." + }, + { + "id": "70178", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 23." + }, + { + "id": "70179", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 24." + }, + { + "id": "70180", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 25." + }, + { + "id": "70181", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 26." + }, + { + "id": "70182", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 27." + }, + { + "id": "70183", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 28." + }, + { + "id": "70184", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 29." + }, + { + "id": "70185", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 30." + }, + { + "id": "70186", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 31." + }, + { + "id": "70187", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 32." + }, + { + "id": "70188", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 33." + }, + { + "id": "70189", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 34." + }, + { + "id": "70190", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 35." + }, + { + "id": "70191", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 36." + }, + { + "id": "70192", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 37." + }, + { + "id": "70193", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 38." + }, + { + "id": "70194", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 39." + }, + { + "id": "70195", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 40." + }, + { + "id": "70196", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 41." + }, + { + "id": "70197", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 42." + }, + { + "id": "70198", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 43." + }, + { + "id": "70199", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 44." + }, + { + "id": "70200", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 45." + }, + { + "id": "70201", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 46." + }, + { + "id": "70202", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 47." + }, + { + "id": "70203", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 48." + }, + { + "id": "70204", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 49." + }, + { + "id": "70205", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 50." + }, + { + "id": "70206", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 51." + }, + { + "id": "70207", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 52." + }, + { + "id": "70208", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 53." + }, + { + "id": "70209", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 54." + }, + { + "id": "70210", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 55." + }, + { + "id": "70211", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 56." + }, + { + "id": "70212", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 57." + }, + { + "id": "70213", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 58." + }, + { + "id": "70214", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 59." + }, + { + "id": "70215", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 60." + }, + { + "id": "70216", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 61." + }, + { + "id": "70217", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 62." + }, + { + "id": "70218", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 63." + }, + { + "id": "70219", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 64." + }, + { + "id": "70220", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 65." + }, + { + "id": "70221", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 66." + }, + { + "id": "70222", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 67." + }, + { + "id": "70223", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 68." + }, + { + "id": "70224", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 69." + }, + { + "id": "70225", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 70." + }, + { + "id": "70226", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 71." + }, + { + "id": "70227", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 72." + }, + { + "id": "70228", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 73." + }, + { + "id": "70229", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 74." + }, + { + "id": "70230", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 75." + }, + { + "id": "70231", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 76." + }, + { + "id": "70232", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 77." + }, + { + "id": "70233", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 78." + }, + { + "id": "70234", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 79." + }, + { + "id": "70235", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 80." + }, + { + "id": "70236", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 81." + }, + { + "id": "70237", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 82." + }, + { + "id": "70238", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 83." + }, + { + "id": "70239", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 84." + }, + { + "id": "70240", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 85." + }, + { + "id": "70241", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 86." + }, + { + "id": "70242", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 87." + }, + { + "id": "70243", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 88." + }, + { + "id": "70244", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 89." + }, + { + "id": "70245", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 90." + }, + { + "id": "70246", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 91." + }, + { + "id": "70247", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 92." + }, + { + "id": "70248", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 93." + }, + { + "id": "70249", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 94." + }, + { + "id": "70250", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 95." + }, + { + "id": "70251", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 96." + }, + { + "id": "70252", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 97." + }, + { + "id": "70253", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 98." + }, + { + "id": "70254", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 99." + }, + { + "id": "70255", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 100." + }, + { + "id": "70256", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 101." + }, + { + "id": "70257", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 102." + }, + { + "id": "70258", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 103." + }, + { + "id": "70259", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 104." + }, + { + "id": "70260", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 105." + }, + { + "id": "70261", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 106." + }, + { + "id": "70262", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 107." + }, + { + "id": "70263", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 108." + }, + { + "id": "70264", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 109." + }, + { + "id": "70265", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 110." + }, + { + "id": "70266", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 111." + }, + { + "id": "70267", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 112." + }, + { + "id": "70268", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 113." + }, + { + "id": "70269", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 114." + }, + { + "id": "70270", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 115." + }, + { + "id": "70271", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 116." + }, + { + "id": "70272", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 117." + }, + { + "id": "70273", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 118." + }, + { + "id": "70274", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 119." + }, + { + "id": "70275", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 120." + }, + { + "id": "70276", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 121." + }, + { + "id": "70277", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 122." + }, + { + "id": "70278", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 123." + }, + { + "id": "70279", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 124." + }, + { + "id": "70280", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 125." + }, + { + "id": "70281", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 126." + }, + { + "id": "70282", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 127." + }, + { + "id": "70283", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 128." + }, + { + "id": "70284", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 129." + }, + { + "id": "70285", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 130." + }, + { + "id": "70286", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 131." + }, + { + "id": "70287", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 132." + }, + { + "id": "70288", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 133." + }, + { + "id": "70289", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 134." + }, + { + "id": "70290", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 135." + }, + { + "id": "70291", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 136." + }, + { + "id": "70292", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 137." + }, + { + "id": "70293", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 138." + }, + { + "id": "70294", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 139." + }, + { + "id": "70295", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 140." + }, + { + "id": "70296", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 141." + }, + { + "id": "70297", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 142." + }, + { + "id": "70298", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 143." + }, + { + "id": "70299", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 144." + }, + { + "id": "70300", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 145." + }, + { + "id": "70301", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 146." + }, + { + "id": "70302", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch export encryption: empty form awaiting the actual policy. Item 147." + }, + { + "id": "70303", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 148." + }, + { + "id": "70304", + "chatId": "202", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 149." + }, + { + "id": "70305", + "chatId": "203", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Спектр очередь лимит согласован: принимать не более 120 задач, остальные отправлять в резервную очередь." + }, + { + "id": "70306", + "chatId": "203", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Спектр очередь лимит подтвержден, владелец Даша." + }, + { + "id": "70307", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 0." + }, + { + "id": "70308", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 1." + }, + { + "id": "70309", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 2." + }, + { + "id": "70310", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 3." + }, + { + "id": "70311", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 4." + }, + { + "id": "70312", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 5." + }, + { + "id": "70313", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 6." + }, + { + "id": "70314", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 7." + }, + { + "id": "70315", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 8." + }, + { + "id": "70316", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 9." + }, + { + "id": "70317", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 10." + }, + { + "id": "70318", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 11." + }, + { + "id": "70319", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 12." + }, + { + "id": "70320", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 13." + }, + { + "id": "70321", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 14." + }, + { + "id": "70322", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 15." + }, + { + "id": "70323", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 16." + }, + { + "id": "70324", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 17." + }, + { + "id": "70325", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 18." + }, + { + "id": "70326", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 19." + }, + { + "id": "70327", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 20." + }, + { + "id": "70328", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 21." + }, + { + "id": "70329", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 22." + }, + { + "id": "70330", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 23." + }, + { + "id": "70331", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 24." + }, + { + "id": "70332", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 25." + }, + { + "id": "70333", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 26." + }, + { + "id": "70334", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 27." + }, + { + "id": "70335", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 28." + }, + { + "id": "70336", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 29." + }, + { + "id": "70337", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 30." + }, + { + "id": "70338", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 31." + }, + { + "id": "70339", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 32." + }, + { + "id": "70340", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 33." + }, + { + "id": "70341", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 34." + }, + { + "id": "70342", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 35." + }, + { + "id": "70343", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 36." + }, + { + "id": "70344", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 37." + }, + { + "id": "70345", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 38." + }, + { + "id": "70346", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 39." + }, + { + "id": "70347", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 40." + }, + { + "id": "70348", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 41." + }, + { + "id": "70349", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 42." + }, + { + "id": "70350", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 43." + }, + { + "id": "70351", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 44." + }, + { + "id": "70352", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 45." + }, + { + "id": "70353", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 46." + }, + { + "id": "70354", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 47." + }, + { + "id": "70355", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 48." + }, + { + "id": "70356", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 49." + }, + { + "id": "70357", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 50." + }, + { + "id": "70358", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 51." + }, + { + "id": "70359", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 52." + }, + { + "id": "70360", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 53." + }, + { + "id": "70361", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 54." + }, + { + "id": "70362", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 55." + }, + { + "id": "70363", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 56." + }, + { + "id": "70364", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 57." + }, + { + "id": "70365", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 58." + }, + { + "id": "70366", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 59." + }, + { + "id": "70367", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 60." + }, + { + "id": "70368", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 61." + }, + { + "id": "70369", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 62." + }, + { + "id": "70370", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 63." + }, + { + "id": "70371", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 64." + }, + { + "id": "70372", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 65." + }, + { + "id": "70373", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 66." + }, + { + "id": "70374", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 67." + }, + { + "id": "70375", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 68." + }, + { + "id": "70376", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 69." + }, + { + "id": "70377", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 70." + }, + { + "id": "70378", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 71." + }, + { + "id": "70379", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 72." + }, + { + "id": "70380", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 73." + }, + { + "id": "70381", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 74." + }, + { + "id": "70382", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 75." + }, + { + "id": "70383", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 76." + }, + { + "id": "70384", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 77." + }, + { + "id": "70385", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 78." + }, + { + "id": "70386", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 79." + }, + { + "id": "70387", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 80." + }, + { + "id": "70388", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 81." + }, + { + "id": "70389", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 82." + }, + { + "id": "70390", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 83." + }, + { + "id": "70391", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 84." + }, + { + "id": "70392", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 85." + }, + { + "id": "70393", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 86." + }, + { + "id": "70394", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 87." + }, + { + "id": "70395", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 88." + }, + { + "id": "70396", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 89." + }, + { + "id": "70397", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 90." + }, + { + "id": "70398", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 91." + }, + { + "id": "70399", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 92." + }, + { + "id": "70400", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 93." + }, + { + "id": "70401", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 94." + }, + { + "id": "70402", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 95." + }, + { + "id": "70403", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 96." + }, + { + "id": "70404", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 97." + }, + { + "id": "70405", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 98." + }, + { + "id": "70406", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 99." + }, + { + "id": "70407", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 100." + }, + { + "id": "70408", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 101." + }, + { + "id": "70409", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 102." + }, + { + "id": "70410", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 103." + }, + { + "id": "70411", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 104." + }, + { + "id": "70412", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 105." + }, + { + "id": "70413", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 106." + }, + { + "id": "70414", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 107." + }, + { + "id": "70415", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 108." + }, + { + "id": "70416", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 109." + }, + { + "id": "70417", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 110." + }, + { + "id": "70418", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 111." + }, + { + "id": "70419", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 112." + }, + { + "id": "70420", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 113." + }, + { + "id": "70421", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 114." + }, + { + "id": "70422", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 115." + }, + { + "id": "70423", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 116." + }, + { + "id": "70424", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 117." + }, + { + "id": "70425", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 118." + }, + { + "id": "70426", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 119." + }, + { + "id": "70427", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 120." + }, + { + "id": "70428", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 121." + }, + { + "id": "70429", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 122." + }, + { + "id": "70430", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 123." + }, + { + "id": "70431", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 124." + }, + { + "id": "70432", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 125." + }, + { + "id": "70433", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 126." + }, + { + "id": "70434", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 127." + }, + { + "id": "70435", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 128." + }, + { + "id": "70436", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 129." + }, + { + "id": "70437", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 130." + }, + { + "id": "70438", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 131." + }, + { + "id": "70439", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 132." + }, + { + "id": "70440", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 133." + }, + { + "id": "70441", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 134." + }, + { + "id": "70442", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 135." + }, + { + "id": "70443", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 136." + }, + { + "id": "70444", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 137." + }, + { + "id": "70445", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 138." + }, + { + "id": "70446", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 139." + }, + { + "id": "70447", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 140." + }, + { + "id": "70448", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 141." + }, + { + "id": "70449", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 142." + }, + { + "id": "70450", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 143." + }, + { + "id": "70451", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 144." + }, + { + "id": "70452", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 145." + }, + { + "id": "70453", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 146." + }, + { + "id": "70454", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 147." + }, + { + "id": "70455", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 148." + }, + { + "id": "70456", + "chatId": "203", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 149." + }, + { + "id": "70457", + "chatId": "204", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Nova access expiry policy: expire temporary access after 48 hours and require a new approval to extend it." + }, + { + "id": "70458", + "chatId": "204", + "senderId": "800", + "timestamp": "2026-10-05T12:00:00.000Z", + "text": "Nova access expiry owner is Bo; the agreed policy is documented." + }, + { + "id": "70459", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 0." + }, + { + "id": "70460", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 1." + }, + { + "id": "70461", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 2." + }, + { + "id": "70462", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 3." + }, + { + "id": "70463", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 4." + }, + { + "id": "70464", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 5." + }, + { + "id": "70465", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 6." + }, + { + "id": "70466", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 7." + }, + { + "id": "70467", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 8." + }, + { + "id": "70468", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 9." + }, + { + "id": "70469", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 10." + }, + { + "id": "70470", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 11." + }, + { + "id": "70471", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 12." + }, + { + "id": "70472", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 13." + }, + { + "id": "70473", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 14." + }, + { + "id": "70474", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 15." + }, + { + "id": "70475", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 16." + }, + { + "id": "70476", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 17." + }, + { + "id": "70477", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 18." + }, + { + "id": "70478", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 19." + }, + { + "id": "70479", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 20." + }, + { + "id": "70480", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 21." + }, + { + "id": "70481", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 22." + }, + { + "id": "70482", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 23." + }, + { + "id": "70483", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 24." + }, + { + "id": "70484", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 25." + }, + { + "id": "70485", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 26." + }, + { + "id": "70486", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 27." + }, + { + "id": "70487", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 28." + }, + { + "id": "70488", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 29." + }, + { + "id": "70489", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 30." + }, + { + "id": "70490", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 31." + }, + { + "id": "70491", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 32." + }, + { + "id": "70492", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 33." + }, + { + "id": "70493", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 34." + }, + { + "id": "70494", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 35." + }, + { + "id": "70495", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 36." + }, + { + "id": "70496", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 37." + }, + { + "id": "70497", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 38." + }, + { + "id": "70498", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 39." + }, + { + "id": "70499", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 40." + }, + { + "id": "70500", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 41." + }, + { + "id": "70501", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 42." + }, + { + "id": "70502", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 43." + }, + { + "id": "70503", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 44." + }, + { + "id": "70504", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 45." + }, + { + "id": "70505", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 46." + }, + { + "id": "70506", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 47." + }, + { + "id": "70507", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 48." + }, + { + "id": "70508", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 49." + }, + { + "id": "70509", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 50." + }, + { + "id": "70510", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 51." + }, + { + "id": "70511", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 52." + }, + { + "id": "70512", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 53." + }, + { + "id": "70513", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 54." + }, + { + "id": "70514", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 55." + }, + { + "id": "70515", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 56." + }, + { + "id": "70516", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 57." + }, + { + "id": "70517", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 58." + }, + { + "id": "70518", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 59." + }, + { + "id": "70519", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 60." + }, + { + "id": "70520", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 61." + }, + { + "id": "70521", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 62." + }, + { + "id": "70522", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 63." + }, + { + "id": "70523", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 64." + }, + { + "id": "70524", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 65." + }, + { + "id": "70525", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 66." + }, + { + "id": "70526", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 67." + }, + { + "id": "70527", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 68." + }, + { + "id": "70528", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 69." + }, + { + "id": "70529", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 70." + }, + { + "id": "70530", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 71." + }, + { + "id": "70531", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 72." + }, + { + "id": "70532", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 73." + }, + { + "id": "70533", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 74." + }, + { + "id": "70534", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 75." + }, + { + "id": "70535", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 76." + }, + { + "id": "70536", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 77." + }, + { + "id": "70537", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 78." + }, + { + "id": "70538", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 79." + }, + { + "id": "70539", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 80." + }, + { + "id": "70540", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 81." + }, + { + "id": "70541", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 82." + }, + { + "id": "70542", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 83." + }, + { + "id": "70543", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 84." + }, + { + "id": "70544", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 85." + }, + { + "id": "70545", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 86." + }, + { + "id": "70546", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 87." + }, + { + "id": "70547", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 88." + }, + { + "id": "70548", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 89." + }, + { + "id": "70549", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 90." + }, + { + "id": "70550", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 91." + }, + { + "id": "70551", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 92." + }, + { + "id": "70552", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 93." + }, + { + "id": "70553", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 94." + }, + { + "id": "70554", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 95." + }, + { + "id": "70555", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 96." + }, + { + "id": "70556", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 97." + }, + { + "id": "70557", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 98." + }, + { + "id": "70558", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 99." + }, + { + "id": "70559", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 100." + }, + { + "id": "70560", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 101." + }, + { + "id": "70561", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 102." + }, + { + "id": "70562", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 103." + }, + { + "id": "70563", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 104." + }, + { + "id": "70564", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 105." + }, + { + "id": "70565", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 106." + }, + { + "id": "70566", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 107." + }, + { + "id": "70567", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 108." + }, + { + "id": "70568", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 109." + }, + { + "id": "70569", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 110." + }, + { + "id": "70570", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 111." + }, + { + "id": "70571", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 112." + }, + { + "id": "70572", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 113." + }, + { + "id": "70573", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 114." + }, + { + "id": "70574", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 115." + }, + { + "id": "70575", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 116." + }, + { + "id": "70576", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 117." + }, + { + "id": "70577", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 118." + }, + { + "id": "70578", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 119." + }, + { + "id": "70579", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 120." + }, + { + "id": "70580", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 121." + }, + { + "id": "70581", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 122." + }, + { + "id": "70582", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 123." + }, + { + "id": "70583", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 124." + }, + { + "id": "70584", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 125." + }, + { + "id": "70585", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 126." + }, + { + "id": "70586", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 127." + }, + { + "id": "70587", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 128." + }, + { + "id": "70588", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 129." + }, + { + "id": "70589", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 130." + }, + { + "id": "70590", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 131." + }, + { + "id": "70591", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 132." + }, + { + "id": "70592", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 133." + }, + { + "id": "70593", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 134." + }, + { + "id": "70594", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 135." + }, + { + "id": "70595", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 136." + }, + { + "id": "70596", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 137." + }, + { + "id": "70597", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 138." + }, + { + "id": "70598", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 139." + }, + { + "id": "70599", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 140." + }, + { + "id": "70600", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 141." + }, + { + "id": "70601", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 142." + }, + { + "id": "70602", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 143." + }, + { + "id": "70603", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 144." + }, + { + "id": "70604", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 145." + }, + { + "id": "70605", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 146." + }, + { + "id": "70606", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova access expiry: empty form awaiting the actual policy. Item 147." + }, + { + "id": "70607", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 148." + }, + { + "id": "70608", + "chatId": "204", + "senderId": "801", + "timestamp": "2026-10-06T12:00:00.000Z", + "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 149." + } + ], + "queries": [ + { + "id": "aurora-question-decision", + "split": "dev", + "family": "aurora", + "category": "decision", + "retrieval": "aurora rollout rollback", + "text": "What rollback decision was agreed for the Aurora rollout?", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-question-question", + "split": "dev", + "family": "aurora", + "category": "question", + "retrieval": "aurora rollout rollback", + "text": "What should we do to roll back the Aurora rollout?", + "relevant": { + "10001": 2, + "10002": 1 + } + }, + { + "id": "aurora-question-no-answer", + "split": "dev", + "family": "aurora", + "category": "no-answer", + "retrieval": "aurora rollout rollback", + "text": "Which independent auditor signed off on aurora rollout rollback?", + "relevant": {} + }, + { + "id": "cedar-question-decision", + "split": "dev", + "family": "cedar", + "category": "decision", + "retrieval": "cedar backup retention", + "text": "What retention policy was agreed for Cedar backups?", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-question-question", + "split": "dev", + "family": "cedar", + "category": "question", + "retrieval": "cedar backup retention", + "text": "How long should Cedar backup snapshots be kept?", + "relevant": { + "10303": 2, + "10304": 1 + } + }, + { + "id": "cedar-question-no-answer", + "split": "dev", + "family": "cedar", + "category": "no-answer", + "retrieval": "cedar backup retention", + "text": "Which independent auditor signed off on cedar backup retention?", + "relevant": {} + }, + { + "id": "север-question-decision", + "split": "dev", + "family": "север", + "category": "decision", + "retrieval": "север сертификат обновление", + "text": "Какое решение принято по обновлению сертификата Север?", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-question-question", + "split": "dev", + "family": "север", + "category": "question", + "retrieval": "север сертификат обновление", + "text": "Что известно о новом сертификате Север и сроке его действия?", + "relevant": { + "10605": 2, + "10606": 1 + } + }, + { + "id": "север-question-no-answer", + "split": "dev", + "family": "север", + "category": "no-answer", + "retrieval": "север сертификат обновление", + "text": "Какой независимый аудитор подписал обновление сертификата Север?", + "relevant": {} + }, + { + "id": "lumen-question-decision", + "split": "dev", + "family": "lumen", + "category": "decision", + "retrieval": "lumen invoice refund", + "text": "What refund was approved for the Lumen invoice?", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-question-question", + "split": "dev", + "family": "lumen", + "category": "question", + "retrieval": "lumen invoice refund", + "text": "What should be returned to the payer of the Lumen invoice?", + "relevant": { + "10907": 2, + "10908": 1 + } + }, + { + "id": "lumen-question-no-answer", + "split": "dev", + "family": "lumen", + "category": "no-answer", + "retrieval": "lumen invoice refund", + "text": "Which independent auditor signed off on lumen invoice refund?", + "relevant": {} + }, + { + "id": "cobalt-question-decision", + "split": "test", + "family": "cobalt", + "category": "decision", + "retrieval": "cobalt failover routing", + "text": "What failover routing was approved for Cobalt?", + "relevant": { + "70001": 2, + "70002": 1 + } + }, + { + "id": "cobalt-question-question", + "split": "test", + "family": "cobalt", + "category": "question", + "retrieval": "cobalt failover routing", + "text": "Where should Cobalt traffic go after failover?", + "relevant": { + "70001": 2, + "70002": 1 + } + }, + { + "id": "cobalt-question-no-answer", + "split": "test", + "family": "cobalt", + "category": "no-answer", + "retrieval": "cobalt failover routing", + "text": "Which independent auditor signed off on cobalt failover routing?", + "relevant": {} + }, + { + "id": "birch-question-decision", + "split": "test", + "family": "birch", + "category": "decision", + "retrieval": "birch export encryption", + "text": "What export encryption policy was agreed for Birch?", + "relevant": { + "70153": 2, + "70154": 1 + } + }, + { + "id": "birch-question-question", + "split": "test", + "family": "birch", + "category": "question", + "retrieval": "birch export encryption", + "text": "How should Birch exports be encrypted before they are sent?", + "relevant": { + "70153": 2, + "70154": 1 + } + }, + { + "id": "birch-question-no-answer", + "split": "test", + "family": "birch", + "category": "no-answer", + "retrieval": "birch export encryption", + "text": "Which independent auditor signed off on birch export encryption?", + "relevant": {} + }, + { + "id": "спектр-question-decision", + "split": "test", + "family": "спектр", + "category": "decision", + "retrieval": "спектр очередь лимит", + "text": "Какой лимит очереди согласован для Спектр?", + "relevant": { + "70305": 2, + "70306": 1 + } + }, + { + "id": "спектр-question-question", + "split": "test", + "family": "спектр", + "category": "question", + "retrieval": "спектр очередь лимит", + "text": "Сколько задач принимает очередь Спектр и куда отправлять остальные?", + "relevant": { + "70305": 2, + "70306": 1 + } + }, + { + "id": "спектр-question-no-answer", + "split": "test", + "family": "спектр", + "category": "no-answer", + "retrieval": "спектр очередь лимит", + "text": "Какой независимый аудитор подписал лимит очереди Спектр?", + "relevant": {} + }, + { + "id": "nova-question-decision", + "split": "test", + "family": "nova", + "category": "decision", + "retrieval": "nova access expiry", + "text": "What access expiry policy was agreed for Nova?", + "relevant": { + "70457": 2, + "70458": 1 + } + }, + { + "id": "nova-question-question", + "split": "test", + "family": "nova", + "category": "question", + "retrieval": "nova access expiry", + "text": "When does Nova temporary access expire and how can it be extended?", + "relevant": { + "70457": 2, + "70458": 1 + } + }, + { + "id": "nova-question-no-answer", + "split": "test", + "family": "nova", + "category": "no-answer", + "retrieval": "nova access expiry", + "text": "Which independent auditor signed off on nova access expiry?", + "relevant": {} + } + ] +} diff --git a/bench/message-search-quality/reranker.json b/bench/message-search-quality/reranker.json new file mode 100644 index 000000000..7e6f1908d --- /dev/null +++ b/bench/message-search-quality/reranker.json @@ -0,0 +1,28 @@ +{ + "repository": "cross-encoder/mmarco-mMiniLMv2-L12-H384-v1", + "revision": "1427fd652930e4ba29e8149678df786c240d8825", + "licence": "Apache-2.0", + "maxTokens": 512, + "files": [ + { + "name": "onnx/model_quint8_avx2.onnx", + "sha256": "6c2513767fb63d008a4377bef7a7a3555433d9436342bb53e35a3a72ffc52d4b", + "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/onnx/model_quint8_avx2.onnx" + }, + { + "name": "tokenizer.json", + "sha256": "62c24cdc13d4c9952d63718d6c9fa4c287974249e16b7ade6d5a85e7bbb75626", + "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/tokenizer.json" + }, + { + "name": "tokenizer_config.json", + "sha256": "e7fbfbfa6347b4e414c1cee50d142e2c2f9a895dad68b068ae83a8b564c3837e", + "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/tokenizer_config.json" + }, + { + "name": "config.json", + "sha256": "cc2cfe51aa3fd759d21d21acf5dfd6994aa67a3c9210636d22e143699d336c77", + "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/config.json" + } + ] +} diff --git a/bench/message-search-quality/run.ts b/bench/message-search-quality/run.ts index 37482ed7d..6f8e83bb8 100644 --- a/bench/message-search-quality/run.ts +++ b/bench/message-search-quality/run.ts @@ -40,6 +40,12 @@ type Row = { const corpusBytes = readFileSync(new URL("./corpus.json", import.meta.url)) const corpus = JSON.parse(corpusBytes.toString()) as Corpus +const supplementBytes = process.argv.includes("--include-questions") ? readFileSync(new URL("./questions.json", import.meta.url)) : undefined +if (supplementBytes) { + const supplement = JSON.parse(supplementBytes.toString()) as Pick + corpus.chats.push(...supplement.chats) + corpus.messages.push(...supplement.messages) +} const ids = new Set(corpus.messages.map(({ id }) => id)) assert.equal(ids.size, corpus.messages.length) assert.equal(new Set(corpus.queries.map(({ id }) => id)).size, corpus.queries.length) @@ -115,6 +121,7 @@ try { "search/correct.js", "search/stem.js", "search/lucene/parser.js", "store/sqlite/lucene.js", "store/sqlite/words.js"] console.log(JSON.stringify({ schemaVersion: 1, k: 10, node: process.version, timezone: "UTC", storePath: path, + archiveSupplementSha256: supplementBytes ? digest(supplementBytes) : null, corpusSha256: digest(corpusBytes), runnerSha256: digest(readFileSync(fileURLToPath(import.meta.url))), productionSha256: Object.fromEntries(productionFiles.map((file) => [file, digest(readFileSync(new URL(`../../dist/${file}`, import.meta.url)))])), messages: corpus.messages.length, queries: corpus.queries.length, diff --git a/bench/message-search-quality/semantic-dev.json b/bench/message-search-quality/semantic-dev.json new file mode 100644 index 000000000..2adec9716 --- /dev/null +++ b/bench/message-search-quality/semantic-dev.json @@ -0,0 +1,27423 @@ +{ + "crossEncoderImplementationSha256": "2d044b35dc1f4ac769127e1b86303b5b4d20f7970eb2ea9a47df729153e96aa0", + "rerankerManifestSha256": "8efb505319d19cb42772dc02438cd99c68cfd4771bcba8890aac3ec81b1538d3", + "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", + "questionsSha256": "390ff1475964d982a06afaf9e2f6f840f5b8304fcffe58ebedd5a0ea8d54478e", + "runnerSha256": "bea007961d0dc1d237824c274029f81dce28c4840e8df88c9b2b282d9bf07fd5", + "productionSha256": { + "services/messages-combined.js": "d10aeb5f7049f615503d59c37df109bcd284f53b5cfceaac0151f2d7e2ca60e9", + "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", + "embeddings/embed.js": "cacf0bec8b98aeda4ed135642d593c4d5b50800867bc8e95de0ad77c1e002e3f", + "embeddings/models.js": "8debfaa01509728ba206702f27e6d8251b8968a03375089b6ca1af9d0eb53e94", + "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964" + }, + "node": "v24.19.0", + "cpu": "AMD Ryzen AI 9 HX 470 w/ Radeon 890M", + "storePath": "/tmp/semantic-message-quality-2H4Bdn/quality.db", + "model": { + "id": "e5-small", + "files": [ + { + "name": "onnx/model_quantized.onnx", + "sha256": "f80102d3f2a1229f387d3c81909990d8945513e347b0eab049f7de3c6f98c193" + }, + { + "name": "tokenizer.json", + "sha256": "0b44a9d7b51c3c62626640cda0e2c2f70fdacdc25bbbd68038369d14ebdf4c39" + }, + { + "name": "tokenizer_config.json", + "sha256": "a1d6bc8734a6f635dc158508bef000f8e2e5a759c7d92f984b2c86e5ff53425b" + } + ], + "threads": 8 + }, + "kind": "embedding-cosine reranking; not a cross-encoder", + "candidateSummary": [ + { + "depth": 50, + "candidateRecall": 0.4375, + "byCategory": { + "words": 0.25, + "typo": 0.25, + "beginnings": 0.25, + "phrase": 0.125, + "from": 1, + "chat": 1, + "date": 1, + "and": 0.25, + "or": 0.25, + "paraphrase": 0 + } + }, + { + "depth": 100, + "candidateRecall": 0.75, + "byCategory": { + "words": 0.75, + "typo": 0.75, + "beginnings": 0.75, + "phrase": 0.75, + "from": 1, + "chat": 1, + "date": 1, + "and": 0.75, + "or": 0.75, + "paraphrase": 0 + } + }, + { + "depth": 300, + "candidateRecall": 0.9, + "byCategory": { + "words": 1, + "typo": 1, + "beginnings": 1, + "phrase": 1, + "from": 1, + "chat": 1, + "date": 1, + "and": 1, + "or": 1, + "paraphrase": 0 + } + }, + { + "depth": 500, + "candidateRecall": 0.9, + "byCategory": { + "words": 1, + "typo": 1, + "beginnings": 1, + "phrase": 1, + "from": 1, + "chat": 1, + "date": 1, + "and": 1, + "or": 1, + "paraphrase": 0 + } + } + ], + "selectedOnDev": { + "depth": 300, + "questionScoreFloor": 0.9 + }, + "keywordSummary": { + "lexical": { + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.3625, + "mrr": 0.375, + "ndcgAt10": 0.3263914259579807, + "noAnswerFalseHits": 0 + }, + "semantic": { + "answerable": 40, + "noAnswerQueries": 8, + "recallAt10": 0.275, + "mrr": 0.3, + "ndcgAt10": 0.26737653428714403, + "noAnswerFalseHits": 0 + } + }, + "questionSummary": { + "direct": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + }, + "anchoredLexical": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.25, + "mrr": 0.25, + "ndcgAt10": 0.19917689524762666, + "noAnswerFalseHits": 40 + }, + "sweep": [ + { + "floor": null, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.625, + "ndcgAt10": 0.5963853267110182, + "noAnswerFalseHits": 40 + } + }, + { + "floor": 0.7, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.625, + "ndcgAt10": 0.5963853267110182, + "noAnswerFalseHits": 40 + } + }, + { + "floor": 0.75, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.625, + "ndcgAt10": 0.5963853267110182, + "noAnswerFalseHits": 40 + } + }, + { + "floor": 0.8, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.625, + "ndcgAt10": 0.5963853267110182, + "noAnswerFalseHits": 40 + } + }, + { + "floor": 0.85, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0.5625, + "mrr": 0.625, + "ndcgAt10": 0.5963853267110182, + "noAnswerFalseHits": 35 + } + }, + { + "floor": 0.9, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + } + }, + { + "floor": 0.95, + "metrics": { + "answerable": 8, + "noAnswerQueries": 4, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "noAnswerFalseHits": 0 + } + } + ] + }, + "devQualified": false, + "questionRetrieval": "manual topic anchors for diagnostic reranking; full-question direct retrieval is measured separately", + "heldOut": null, + "inference": { + "pairInferenceMs": 0, + "scoredPairs": 0, + "averagePairMs": null, + "loadMs": 1107.57413, + "embeddedMessages": 1200, + "uniqueEmbeddedQueries": 48, + "uniqueCrossEncoderQueries": 0, + "passageInferenceMs": 12019.547399, + "queryInferenceMs": 356.87080400000104, + "amortizedPerMessageMs": 10.016289499166666, + "cache": "new in-memory synthetic message vectors; no persistent production message index" + }, + "candidateRows": [ + { + "id": "aurora-words", + "category": "words", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-words", + "category": "words", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-words", + "category": "words", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-words", + "category": "words", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 50, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 100, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 300, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-from", + "category": "from", + "depth": 500, + "relevant": { + "10001": 2 + }, + "candidateIds": [ + "10001" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-date", + "category": "date", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10001", + "10002" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-and", + "category": "and", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-and", + "category": "and", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-and", + "category": "and", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-and", + "category": "and", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-or", + "category": "or", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-or", + "category": "or", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-or", + "category": "or", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "aurora-or", + "category": "or", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [ + "10298", + "10293", + "10288", + "10283", + "10278", + "10273", + "10268", + "10263", + "10258", + "10253", + "10248", + "10243", + "10238", + "10233", + "10228", + "10223", + "10218", + "10213", + "10208", + "10203", + "10198", + "10193", + "10188", + "10183", + "10178", + "10173", + "10168", + "10163", + "10158", + "10153", + "10148", + "10143", + "10138", + "10133", + "10128", + "10123", + "10118", + "10113", + "10108", + "10103", + "10098", + "10093", + "10088", + "10083", + "10078", + "10073", + "10068", + "10063", + "10058", + "10053", + "10048", + "10043", + "10038", + "10033", + "10028", + "10023", + "10018", + "10013", + "10008", + "10003", + "10001", + "10299", + "10294", + "10289", + "10284", + "10279", + "10274", + "10269", + "10264", + "10259", + "10254", + "10249", + "10244", + "10239", + "10234", + "10229", + "10224", + "10219", + "10214", + "10209", + "10204", + "10199", + "10194", + "10189", + "10184", + "10179", + "10174", + "10169", + "10164", + "10159", + "10154", + "10149", + "10144", + "10139", + "10134", + "10129", + "10124", + "10119", + "10114", + "10109", + "10104", + "10099", + "10094", + "10089", + "10084", + "10079", + "10074", + "10069", + "10064", + "10059", + "10054", + "10049", + "10044", + "10039", + "10034", + "10029", + "10024", + "10019", + "10014", + "10009", + "10004", + "10002", + "10300", + "10295", + "10290", + "10285", + "10280", + "10275", + "10270", + "10265", + "10260", + "10255", + "10250", + "10245", + "10240", + "10235", + "10230", + "10225", + "10220", + "10215", + "10210", + "10205", + "10200", + "10195", + "10190", + "10185", + "10180", + "10175", + "10170", + "10165", + "10160", + "10155", + "10150", + "10145", + "10140", + "10135", + "10130", + "10125", + "10120", + "10115", + "10110", + "10105", + "10100", + "10095", + "10090", + "10085", + "10080", + "10075", + "10070", + "10065", + "10060", + "10055", + "10050", + "10045", + "10040", + "10035", + "10030", + "10025", + "10020", + "10015", + "10010", + "10005", + "10302", + "10297", + "10292", + "10287", + "10282", + "10277", + "10272", + "10267", + "10262", + "10257", + "10252", + "10247", + "10242", + "10237", + "10232", + "10227", + "10222", + "10217", + "10212", + "10207", + "10202", + "10197", + "10192", + "10187", + "10182", + "10177", + "10172", + "10167", + "10162", + "10157", + "10152", + "10147", + "10142", + "10137", + "10132", + "10127", + "10122", + "10117", + "10112", + "10107", + "10102", + "10097", + "10092", + "10087", + "10082", + "10077", + "10072", + "10067", + "10062", + "10057", + "10052", + "10047", + "10042", + "10037", + "10032", + "10027", + "10022", + "10017", + "10012", + "10007", + "10301", + "10296", + "10291", + "10286", + "10281", + "10276", + "10271", + "10266", + "10261", + "10256", + "10251", + "10246", + "10241", + "10236", + "10231", + "10226", + "10221", + "10216", + "10211", + "10206", + "10201", + "10196", + "10191", + "10186", + "10181", + "10176", + "10171", + "10166", + "10161", + "10156", + "10151", + "10146", + "10141", + "10136", + "10131", + "10126", + "10121", + "10116", + "10111", + "10106", + "10101", + "10096", + "10091", + "10086", + "10081", + "10076", + "10071", + "10066", + "10061", + "10056", + "10051", + "10046", + "10041", + "10036", + "10031", + "10026", + "10021", + "10016", + "10011", + "10006" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10001": 2, + "10002": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-words", + "category": "words", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-words", + "category": "words", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-words", + "category": "words", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-words", + "category": "words", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 50, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 100, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 300, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-from", + "category": "from", + "depth": 500, + "relevant": { + "10303": 2 + }, + "candidateIds": [ + "10303" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-date", + "category": "date", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10303", + "10304" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-and", + "category": "and", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-and", + "category": "and", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-and", + "category": "and", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-and", + "category": "and", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-or", + "category": "or", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-or", + "category": "or", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411" + ], + "candidateRecall": 0.5, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-or", + "category": "or", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "cedar-or", + "category": "or", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [ + "10600", + "10595", + "10590", + "10585", + "10580", + "10575", + "10570", + "10565", + "10560", + "10555", + "10550", + "10545", + "10540", + "10535", + "10530", + "10525", + "10520", + "10515", + "10510", + "10505", + "10500", + "10495", + "10490", + "10485", + "10480", + "10475", + "10470", + "10465", + "10460", + "10455", + "10450", + "10445", + "10440", + "10435", + "10430", + "10425", + "10420", + "10415", + "10410", + "10405", + "10400", + "10395", + "10390", + "10385", + "10380", + "10375", + "10370", + "10365", + "10360", + "10355", + "10350", + "10345", + "10340", + "10335", + "10330", + "10325", + "10320", + "10315", + "10310", + "10305", + "10303", + "10601", + "10596", + "10591", + "10586", + "10581", + "10576", + "10571", + "10566", + "10561", + "10556", + "10551", + "10546", + "10541", + "10536", + "10531", + "10526", + "10521", + "10516", + "10511", + "10506", + "10501", + "10496", + "10491", + "10486", + "10481", + "10476", + "10471", + "10466", + "10461", + "10456", + "10451", + "10446", + "10441", + "10436", + "10431", + "10426", + "10421", + "10416", + "10411", + "10406", + "10401", + "10396", + "10391", + "10386", + "10381", + "10376", + "10371", + "10366", + "10361", + "10356", + "10351", + "10346", + "10341", + "10336", + "10331", + "10326", + "10321", + "10316", + "10311", + "10306", + "10304", + "10602", + "10597", + "10592", + "10587", + "10582", + "10577", + "10572", + "10567", + "10562", + "10557", + "10552", + "10547", + "10542", + "10537", + "10532", + "10527", + "10522", + "10517", + "10512", + "10507", + "10502", + "10497", + "10492", + "10487", + "10482", + "10477", + "10472", + "10467", + "10462", + "10457", + "10452", + "10447", + "10442", + "10437", + "10432", + "10427", + "10422", + "10417", + "10412", + "10407", + "10402", + "10397", + "10392", + "10387", + "10382", + "10377", + "10372", + "10367", + "10362", + "10357", + "10352", + "10347", + "10342", + "10337", + "10332", + "10327", + "10322", + "10317", + "10312", + "10307", + "10604", + "10599", + "10594", + "10589", + "10584", + "10579", + "10574", + "10569", + "10564", + "10559", + "10554", + "10549", + "10544", + "10539", + "10534", + "10529", + "10524", + "10519", + "10514", + "10509", + "10504", + "10499", + "10494", + "10489", + "10484", + "10479", + "10474", + "10469", + "10464", + "10459", + "10454", + "10449", + "10444", + "10439", + "10434", + "10429", + "10424", + "10419", + "10414", + "10409", + "10404", + "10399", + "10394", + "10389", + "10384", + "10379", + "10374", + "10369", + "10364", + "10359", + "10354", + "10349", + "10344", + "10339", + "10334", + "10329", + "10324", + "10319", + "10314", + "10309", + "10603", + "10598", + "10593", + "10588", + "10583", + "10578", + "10573", + "10568", + "10563", + "10558", + "10553", + "10548", + "10543", + "10538", + "10533", + "10528", + "10523", + "10518", + "10513", + "10508", + "10503", + "10498", + "10493", + "10488", + "10483", + "10478", + "10473", + "10468", + "10463", + "10458", + "10453", + "10448", + "10443", + "10438", + "10433", + "10428", + "10423", + "10418", + "10413", + "10408", + "10403", + "10398", + "10393", + "10388", + "10383", + "10378", + "10373", + "10368", + "10363", + "10358", + "10353", + "10348", + "10343", + "10338", + "10333", + "10328", + "10323", + "10318", + "10313", + "10308" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10303": 2, + "10304": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-words", + "category": "words", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-words", + "category": "words", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-words", + "category": "words", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-words", + "category": "words", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662" + ], + "candidateRecall": 0.5, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10605", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10605", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611" + ], + "candidateRecall": 1, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10605", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611" + ], + "candidateRecall": 1, + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 50, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 100, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 300, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-from", + "category": "from", + "depth": 500, + "relevant": { + "10605": 2 + }, + "candidateIds": [ + "10605" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-date", + "category": "date", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10605", + "10606" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-and", + "category": "and", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-and", + "category": "and", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-and", + "category": "and", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-and", + "category": "and", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-or", + "category": "or", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-or", + "category": "or", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-or", + "category": "or", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": true + }, + { + "id": "север-or", + "category": "or", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [ + "10606", + "10605", + "10902", + "10897", + "10892", + "10887", + "10882", + "10877", + "10872", + "10867", + "10862", + "10857", + "10852", + "10847", + "10842", + "10837", + "10832", + "10827", + "10822", + "10817", + "10812", + "10807", + "10802", + "10797", + "10792", + "10787", + "10782", + "10777", + "10772", + "10767", + "10762", + "10757", + "10752", + "10747", + "10742", + "10737", + "10732", + "10727", + "10722", + "10717", + "10712", + "10707", + "10702", + "10697", + "10692", + "10687", + "10682", + "10677", + "10672", + "10667", + "10662", + "10657", + "10652", + "10647", + "10642", + "10637", + "10632", + "10627", + "10622", + "10617", + "10612", + "10607", + "10903", + "10898", + "10893", + "10888", + "10883", + "10878", + "10873", + "10868", + "10863", + "10858", + "10853", + "10848", + "10843", + "10838", + "10833", + "10828", + "10823", + "10818", + "10813", + "10808", + "10803", + "10798", + "10793", + "10788", + "10783", + "10778", + "10773", + "10768", + "10763", + "10758", + "10753", + "10748", + "10743", + "10738", + "10733", + "10728", + "10723", + "10718", + "10713", + "10708", + "10703", + "10698", + "10693", + "10688", + "10683", + "10678", + "10673", + "10668", + "10663", + "10658", + "10653", + "10648", + "10643", + "10638", + "10633", + "10628", + "10623", + "10618", + "10613", + "10608", + "10904", + "10899", + "10894", + "10889", + "10884", + "10879", + "10874", + "10869", + "10864", + "10859", + "10854", + "10849", + "10844", + "10839", + "10834", + "10829", + "10824", + "10819", + "10814", + "10809", + "10804", + "10799", + "10794", + "10789", + "10784", + "10779", + "10774", + "10769", + "10764", + "10759", + "10754", + "10749", + "10744", + "10739", + "10734", + "10729", + "10724", + "10719", + "10714", + "10709", + "10704", + "10699", + "10694", + "10689", + "10684", + "10679", + "10674", + "10669", + "10664", + "10659", + "10654", + "10649", + "10644", + "10639", + "10634", + "10629", + "10624", + "10619", + "10614", + "10609", + "10906", + "10901", + "10896", + "10891", + "10886", + "10881", + "10876", + "10871", + "10866", + "10861", + "10856", + "10851", + "10846", + "10841", + "10836", + "10831", + "10826", + "10821", + "10816", + "10811", + "10806", + "10801", + "10796", + "10791", + "10786", + "10781", + "10776", + "10771", + "10766", + "10761", + "10756", + "10751", + "10746", + "10741", + "10736", + "10731", + "10726", + "10721", + "10716", + "10711", + "10706", + "10701", + "10696", + "10691", + "10686", + "10681", + "10676", + "10671", + "10666", + "10661", + "10656", + "10651", + "10646", + "10641", + "10636", + "10631", + "10626", + "10621", + "10616", + "10611", + "10905", + "10900", + "10895", + "10890", + "10885", + "10880", + "10875", + "10870", + "10865", + "10860", + "10855", + "10850", + "10845", + "10840", + "10835", + "10830", + "10825", + "10820", + "10815", + "10810", + "10805", + "10800", + "10795", + "10790", + "10785", + "10780", + "10775", + "10770", + "10765", + "10760", + "10755", + "10750", + "10745", + "10740", + "10735", + "10730", + "10725", + "10720", + "10715", + "10710", + "10705", + "10700", + "10695", + "10690", + "10685", + "10680", + "10675", + "10670", + "10665", + "10660", + "10655", + "10650", + "10645", + "10640", + "10635", + "10630", + "10625", + "10620", + "10615", + "10610" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10605": 2, + "10606": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-words", + "category": "words", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-words", + "category": "words", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-words", + "category": "words", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-words", + "category": "words", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-typo", + "category": "typo", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-phrase", + "category": "phrase", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 50, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 100, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 300, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-from", + "category": "from", + "depth": 500, + "relevant": { + "10907": 2 + }, + "candidateIds": [ + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-chat", + "category": "chat", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-date", + "category": "date", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "10908", + "10907" + ], + "candidateRecall": 1, + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-and", + "category": "and", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-and", + "category": "and", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-and", + "category": "and", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-and", + "category": "and", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-or", + "category": "or", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959" + ], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-or", + "category": "or", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-or", + "category": "or", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": true + }, + { + "id": "lumen-or", + "category": "or", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [ + "11204", + "11199", + "11194", + "11189", + "11184", + "11179", + "11174", + "11169", + "11164", + "11159", + "11154", + "11149", + "11144", + "11139", + "11134", + "11129", + "11124", + "11119", + "11114", + "11109", + "11104", + "11099", + "11094", + "11089", + "11084", + "11079", + "11074", + "11069", + "11064", + "11059", + "11054", + "11049", + "11044", + "11039", + "11034", + "11029", + "11024", + "11019", + "11014", + "11009", + "11004", + "10999", + "10994", + "10989", + "10984", + "10979", + "10974", + "10969", + "10964", + "10959", + "10954", + "10949", + "10944", + "10939", + "10934", + "10929", + "10924", + "10919", + "10914", + "10909", + "10908", + "10907", + "11205", + "11200", + "11195", + "11190", + "11185", + "11180", + "11175", + "11170", + "11165", + "11160", + "11155", + "11150", + "11145", + "11140", + "11135", + "11130", + "11125", + "11120", + "11115", + "11110", + "11105", + "11100", + "11095", + "11090", + "11085", + "11080", + "11075", + "11070", + "11065", + "11060", + "11055", + "11050", + "11045", + "11040", + "11035", + "11030", + "11025", + "11020", + "11015", + "11010", + "11005", + "11000", + "10995", + "10990", + "10985", + "10980", + "10975", + "10970", + "10965", + "10960", + "10955", + "10950", + "10945", + "10940", + "10935", + "10930", + "10925", + "10920", + "10915", + "10910", + "11206", + "11201", + "11196", + "11191", + "11186", + "11181", + "11176", + "11171", + "11166", + "11161", + "11156", + "11151", + "11146", + "11141", + "11136", + "11131", + "11126", + "11121", + "11116", + "11111", + "11106", + "11101", + "11096", + "11091", + "11086", + "11081", + "11076", + "11071", + "11066", + "11061", + "11056", + "11051", + "11046", + "11041", + "11036", + "11031", + "11026", + "11021", + "11016", + "11011", + "11006", + "11001", + "10996", + "10991", + "10986", + "10981", + "10976", + "10971", + "10966", + "10961", + "10956", + "10951", + "10946", + "10941", + "10936", + "10931", + "10926", + "10921", + "10916", + "10911", + "11208", + "11203", + "11198", + "11193", + "11188", + "11183", + "11178", + "11173", + "11168", + "11163", + "11158", + "11153", + "11148", + "11143", + "11138", + "11133", + "11128", + "11123", + "11118", + "11113", + "11108", + "11103", + "11098", + "11093", + "11088", + "11083", + "11078", + "11073", + "11068", + "11063", + "11058", + "11053", + "11048", + "11043", + "11038", + "11033", + "11028", + "11023", + "11018", + "11013", + "11008", + "11003", + "10998", + "10993", + "10988", + "10983", + "10978", + "10973", + "10968", + "10963", + "10958", + "10953", + "10948", + "10943", + "10938", + "10933", + "10928", + "10923", + "10918", + "10913", + "11207", + "11202", + "11197", + "11192", + "11187", + "11182", + "11177", + "11172", + "11167", + "11162", + "11157", + "11152", + "11147", + "11142", + "11137", + "11132", + "11127", + "11122", + "11117", + "11112", + "11107", + "11102", + "11097", + "11092", + "11087", + "11082", + "11077", + "11072", + "11067", + "11062", + "11057", + "11052", + "11047", + "11042", + "11037", + "11032", + "11027", + "11022", + "11017", + "11012", + "11007", + "11002", + "10997", + "10992", + "10987", + "10982", + "10977", + "10972", + "10967", + "10962", + "10957", + "10952", + "10947", + "10942", + "10937", + "10932", + "10927", + "10922", + "10917", + "10912" + ], + "candidateRecall": 1, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 50, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 100, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 300, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "depth": 500, + "relevant": { + "10907": 2, + "10908": 1 + }, + "candidateIds": [], + "candidateRecall": 0, + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 50, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 100, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 300, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "depth": 500, + "relevant": {}, + "candidateIds": [], + "candidateRecall": null, + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0, + "truncated": false + } + ], + "keywordRows": [ + { + "id": "aurora-words", + "category": "words", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10280", + "score": 0.9192052184191694, + "lexicalRank": 126 + }, + { + "id": "10190", + "score": 0.919062760430714, + "lexicalRank": 144 + }, + { + "id": "10065", + "score": 0.9186915386380173, + "lexicalRank": 169 + }, + { + "id": "10225", + "score": 0.9174911203968827, + "lexicalRank": 137 + }, + { + "id": "10125", + "score": 0.9169708122949837, + "lexicalRank": 157 + }, + { + "id": "10075", + "score": 0.9168015790069396, + "lexicalRank": 167 + }, + { + "id": "10060", + "score": 0.9167733798319815, + "lexicalRank": 170 + }, + { + "id": "10230", + "score": 0.9164214060408934, + "lexicalRank": 136 + }, + { + "id": "10165", + "score": 0.9161771390615109, + "lexicalRank": 149 + }, + { + "id": "10270", + "score": 0.91574480912613, + "lexicalRank": 128 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-typo", + "category": "typo", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10156", + "score": 0.9041150488737639, + "lexicalRank": 271 + }, + { + "id": "10076", + "score": 0.904086260962625, + "lexicalRank": 287 + }, + { + "id": "10166", + "score": 0.9040358962609147, + "lexicalRank": 269 + }, + { + "id": "10065", + "score": 0.903079943144814, + "lexicalRank": 169 + }, + { + "id": "10280", + "score": 0.9028932023824128, + "lexicalRank": 126 + }, + { + "id": "10225", + "score": 0.9028142635467321, + "lexicalRank": 137 + }, + { + "id": "10151", + "score": 0.9024972208063297, + "lexicalRank": 272 + }, + { + "id": "10281", + "score": 0.9022675555094835, + "lexicalRank": 246 + }, + { + "id": "10190", + "score": 0.9021813279751438, + "lexicalRank": 144 + }, + { + "id": "10085", + "score": 0.9020156894563678, + "lexicalRank": 165 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-beginnings", + "category": "beginnings", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10280", + "score": 0.8978866948381442, + "lexicalRank": 126 + }, + { + "id": "10156", + "score": 0.8965793616743678, + "lexicalRank": 271 + }, + { + "id": "10076", + "score": 0.8964982852524149, + "lexicalRank": 287 + }, + { + "id": "10166", + "score": 0.8956208111669557, + "lexicalRank": 269 + }, + { + "id": "10190", + "score": 0.8954033568937041, + "lexicalRank": 144 + }, + { + "id": "10075", + "score": 0.8952151611051157, + "lexicalRank": 167 + }, + { + "id": "10225", + "score": 0.8952053102390007, + "lexicalRank": 137 + }, + { + "id": "10065", + "score": 0.8951843807049377, + "lexicalRank": 169 + }, + { + "id": "10301", + "score": 0.8948241205972687, + "lexicalRank": 242 + }, + { + "id": "10111", + "score": 0.8945984704883156, + "lexicalRank": 280 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-phrase", + "category": "phrase", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 2.7623348535297314 + }, + { + "id": "10293", + "score": 2.7623348535297314 + }, + { + "id": "10288", + "score": 2.7623348535297314 + }, + { + "id": "10283", + "score": 2.7623348535297314 + }, + { + "id": "10278", + "score": 2.7623348535297314 + }, + { + "id": "10273", + "score": 2.7623348535297314 + }, + { + "id": "10268", + "score": 2.7623348535297314 + }, + { + "id": "10263", + "score": 2.7623348535297314 + }, + { + "id": "10258", + "score": 2.7623348535297314 + }, + { + "id": "10253", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "10298", + "score": 2.7623348535297314 + }, + { + "id": "10293", + "score": 2.7623348535297314 + }, + { + "id": "10288", + "score": 2.7623348535297314 + }, + { + "id": "10283", + "score": 2.7623348535297314 + }, + { + "id": "10278", + "score": 2.7623348535297314 + }, + { + "id": "10273", + "score": 2.7623348535297314 + }, + { + "id": "10268", + "score": 2.7623348535297314 + }, + { + "id": "10263", + "score": 2.7623348535297314 + }, + { + "id": "10258", + "score": 2.7623348535297314 + }, + { + "id": "10253", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "aurora-from", + "category": "from", + "relevant": { + "10001": 2 + }, + "lexical": [ + { + "id": "10001", + "score": 1 + } + ], + "semantic": [ + { + "id": "10001", + "score": 0.8873294953955672, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-chat", + "category": "chat", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10223", + "score": 0.8813816418287496, + "lexicalRank": 15 + }, + { + "id": "10263", + "score": 0.8749284164561189, + "lexicalRank": 7 + }, + { + "id": "10233", + "score": 0.8722181190572437, + "lexicalRank": 13 + }, + { + "id": "10268", + "score": 0.8715306928342087, + "lexicalRank": 6 + }, + { + "id": "10218", + "score": 0.8713968455797609, + "lexicalRank": 16 + }, + { + "id": "10208", + "score": 0.8706402050461867, + "lexicalRank": 18 + }, + { + "id": "10253", + "score": 0.870482138126604, + "lexicalRank": 9 + }, + { + "id": "10283", + "score": 0.8698741900246865, + "lexicalRank": 3 + }, + { + "id": "10298", + "score": 0.8697392715074506, + "lexicalRank": 0 + }, + { + "id": "10238", + "score": 0.8696619263428077, + "lexicalRank": 12 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-date", + "category": "date", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10001", + "score": 1 + }, + { + "id": "10002", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10001", + "score": 0.8692216945144376, + "lexicalRank": 0 + }, + { + "id": "10002", + "score": 0.8453639218948915, + "lexicalRank": 1 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-and", + "category": "and", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "aurora-or", + "category": "or", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10298", + "score": 7.489838832938161 + }, + { + "id": "10293", + "score": 7.489838832938161 + }, + { + "id": "10288", + "score": 7.489838832938161 + }, + { + "id": "10283", + "score": 7.489838832938161 + }, + { + "id": "10278", + "score": 7.489838832938161 + }, + { + "id": "10273", + "score": 7.489838832938161 + }, + { + "id": "10268", + "score": 7.489838832938161 + }, + { + "id": "10263", + "score": 7.489838832938161 + }, + { + "id": "10258", + "score": 7.489838832938161 + }, + { + "id": "10253", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "aurora-paraphrase", + "category": "paraphrase", + "relevant": { + "10001": 2, + "10002": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "aurora-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-words", + "category": "words", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10548", + "score": 0.9294003753598464, + "lexicalRank": 253 + }, + { + "id": "10405", + "score": 0.9268079878355586, + "lexicalRank": 39 + }, + { + "id": "10538", + "score": 0.9263857706471276, + "lexicalRank": 255 + }, + { + "id": "10482", + "score": 0.9258707682500287, + "lexicalRank": 146 + }, + { + "id": "10602", + "score": 0.9256494511241109, + "lexicalRank": 122 + }, + { + "id": "10585", + "score": 0.9254712731778304, + "lexicalRank": 3 + }, + { + "id": "10545", + "score": 0.9251230608354597, + "lexicalRank": 11 + }, + { + "id": "10498", + "score": 0.9250750095024941, + "lexicalRank": 263 + }, + { + "id": "10488", + "score": 0.9249750012172294, + "lexicalRank": 265 + }, + { + "id": "10582", + "score": 0.9248633134532314, + "lexicalRank": 126 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-typo", + "category": "typo", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10367", + "score": 0.8806786905837405, + "lexicalRank": 169 + }, + { + "id": "10457", + "score": 0.8802690880138514, + "lexicalRank": 151 + }, + { + "id": "10482", + "score": 0.8795885420606875, + "lexicalRank": 146 + }, + { + "id": "10427", + "score": 0.8788493894056418, + "lexicalRank": 157 + }, + { + "id": "10562", + "score": 0.8784239132925827, + "lexicalRank": 130 + }, + { + "id": "10437", + "score": 0.8777151372157757, + "lexicalRank": 155 + }, + { + "id": "10422", + "score": 0.8773499629723807, + "lexicalRank": 158 + }, + { + "id": "10362", + "score": 0.8771639012061407, + "lexicalRank": 170 + }, + { + "id": "10537", + "score": 0.8769370254072435, + "lexicalRank": 135 + }, + { + "id": "10602", + "score": 0.8767288407478534, + "lexicalRank": 122 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-beginnings", + "category": "beginnings", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10405", + "score": 0.9198652498038274, + "lexicalRank": 39 + }, + { + "id": "10520", + "score": 0.9187540354650561, + "lexicalRank": 16 + }, + { + "id": "10482", + "score": 0.9185876210656136, + "lexicalRank": 146 + }, + { + "id": "10430", + "score": 0.9185690620631375, + "lexicalRank": 34 + }, + { + "id": "10445", + "score": 0.918156502942747, + "lexicalRank": 31 + }, + { + "id": "10602", + "score": 0.9180029649698667, + "lexicalRank": 122 + }, + { + "id": "10427", + "score": 0.917539873668179, + "lexicalRank": 157 + }, + { + "id": "10545", + "score": 0.91736147660192, + "lexicalRank": 11 + }, + { + "id": "10367", + "score": 0.9172398487594343, + "lexicalRank": 169 + }, + { + "id": "10562", + "score": 0.9170926519049367, + "lexicalRank": 130 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-phrase", + "category": "phrase", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 2.7623348535297314 + }, + { + "id": "10595", + "score": 2.7623348535297314 + }, + { + "id": "10590", + "score": 2.7623348535297314 + }, + { + "id": "10585", + "score": 2.7623348535297314 + }, + { + "id": "10580", + "score": 2.7623348535297314 + }, + { + "id": "10575", + "score": 2.7623348535297314 + }, + { + "id": "10570", + "score": 2.7623348535297314 + }, + { + "id": "10565", + "score": 2.7623348535297314 + }, + { + "id": "10560", + "score": 2.7623348535297314 + }, + { + "id": "10555", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "10600", + "score": 2.7623348535297314 + }, + { + "id": "10595", + "score": 2.7623348535297314 + }, + { + "id": "10590", + "score": 2.7623348535297314 + }, + { + "id": "10585", + "score": 2.7623348535297314 + }, + { + "id": "10580", + "score": 2.7623348535297314 + }, + { + "id": "10575", + "score": 2.7623348535297314 + }, + { + "id": "10570", + "score": 2.7623348535297314 + }, + { + "id": "10565", + "score": 2.7623348535297314 + }, + { + "id": "10560", + "score": 2.7623348535297314 + }, + { + "id": "10555", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "cedar-from", + "category": "from", + "relevant": { + "10303": 2 + }, + "lexical": [ + { + "id": "10303", + "score": 1 + } + ], + "semantic": [ + { + "id": "10303", + "score": 0.8648644005237613, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-chat", + "category": "chat", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10510", + "score": 0.8748864283482839, + "lexicalRank": 18 + }, + { + "id": "10595", + "score": 0.8689824008983377, + "lexicalRank": 1 + }, + { + "id": "10525", + "score": 0.8687169265199, + "lexicalRank": 15 + }, + { + "id": "10515", + "score": 0.8684185263437489, + "lexicalRank": 17 + }, + { + "id": "10600", + "score": 0.8659321909233304, + "lexicalRank": 0 + }, + { + "id": "10585", + "score": 0.8658538804618805, + "lexicalRank": 3 + }, + { + "id": "10565", + "score": 0.8651923027273853, + "lexicalRank": 7 + }, + { + "id": "10532", + "score": 0.8649263757798116, + "lexicalRank": 56 + }, + { + "id": "10575", + "score": 0.8646407298812999, + "lexicalRank": 5 + }, + { + "id": "10545", + "score": 0.8641223806525056, + "lexicalRank": 11 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-date", + "category": "date", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10303", + "score": 1 + }, + { + "id": "10304", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10303", + "score": 0.8619150436537566, + "lexicalRank": 0 + }, + { + "id": "10304", + "score": 0.8524625524012509, + "lexicalRank": 1 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-and", + "category": "and", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "cedar-or", + "category": "or", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10600", + "score": 7.489838832938161 + }, + { + "id": "10595", + "score": 7.489838832938161 + }, + { + "id": "10590", + "score": 7.489838832938161 + }, + { + "id": "10585", + "score": 7.489838832938161 + }, + { + "id": "10580", + "score": 7.489838832938161 + }, + { + "id": "10575", + "score": 7.489838832938161 + }, + { + "id": "10570", + "score": 7.489838832938161 + }, + { + "id": "10565", + "score": 7.489838832938161 + }, + { + "id": "10560", + "score": 7.489838832938161 + }, + { + "id": "10555", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "cedar-paraphrase", + "category": "paraphrase", + "relevant": { + "10303": 2, + "10304": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "cedar-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-words", + "category": "words", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10879", + "score": 0.9213124069431595, + "lexicalRank": 127 + }, + { + "id": "10844", + "score": 0.9207854444827525, + "lexicalRank": 134 + }, + { + "id": "10669", + "score": 0.9199201302353809, + "lexicalRank": 169 + }, + { + "id": "10789", + "score": 0.9198138390635751, + "lexicalRank": 145 + }, + { + "id": "10735", + "score": 0.9190050792249866, + "lexicalRank": 276 + }, + { + "id": "10720", + "score": 0.9188604688831258, + "lexicalRank": 279 + }, + { + "id": "10744", + "score": 0.9186825197728935, + "lexicalRank": 154 + }, + { + "id": "10645", + "score": 0.9186674303701673, + "lexicalRank": 294 + }, + { + "id": "10745", + "score": 0.9185491222623472, + "lexicalRank": 274 + }, + { + "id": "10895", + "score": 0.9182719048183844, + "lexicalRank": 244 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-typo", + "category": "typo", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10879", + "score": 0.8985129235259155, + "lexicalRank": 127 + }, + { + "id": "10862", + "score": 0.8962386614132065, + "lexicalRank": 10 + }, + { + "id": "10844", + "score": 0.8955199090239682, + "lexicalRank": 134 + }, + { + "id": "10883", + "score": 0.8951193114737055, + "lexicalRank": 66 + }, + { + "id": "10884", + "score": 0.8948447680739473, + "lexicalRank": 126 + }, + { + "id": "10669", + "score": 0.894776110471853, + "lexicalRank": 169 + }, + { + "id": "10859", + "score": 0.8947304694857922, + "lexicalRank": 131 + }, + { + "id": "10744", + "score": 0.8945010378302453, + "lexicalRank": 154 + }, + { + "id": "10899", + "score": 0.8943928399826122, + "lexicalRank": 123 + }, + { + "id": "10789", + "score": 0.8942916342327378, + "lexicalRank": 145 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-beginnings", + "category": "beginnings", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10879", + "score": 0.8972254388273925, + "lexicalRank": 127 + }, + { + "id": "10674", + "score": 0.8958923841763743, + "lexicalRank": 168 + }, + { + "id": "10789", + "score": 0.8956781566306027, + "lexicalRank": 145 + }, + { + "id": "10844", + "score": 0.8956174612001324, + "lexicalRank": 134 + }, + { + "id": "10744", + "score": 0.8955839035777104, + "lexicalRank": 154 + }, + { + "id": "10659", + "score": 0.8948341665233703, + "lexicalRank": 171 + }, + { + "id": "10669", + "score": 0.8943952104380118, + "lexicalRank": 169 + }, + { + "id": "10709", + "score": 0.8942932783900805, + "lexicalRank": 161 + }, + { + "id": "10799", + "score": 0.8942410899309552, + "lexicalRank": 143 + }, + { + "id": "10884", + "score": 0.8942100783533073, + "lexicalRank": 126 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-phrase", + "category": "phrase", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 3.126652600631594 + }, + { + "id": "10902", + "score": 2.7623348535297314 + }, + { + "id": "10897", + "score": 2.7623348535297314 + }, + { + "id": "10892", + "score": 2.7623348535297314 + }, + { + "id": "10887", + "score": 2.7623348535297314 + }, + { + "id": "10882", + "score": 2.7623348535297314 + }, + { + "id": "10877", + "score": 2.7623348535297314 + }, + { + "id": "10872", + "score": 2.7623348535297314 + }, + { + "id": "10867", + "score": 2.7623348535297314 + }, + { + "id": "10862", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "10606", + "score": 3.126652600631594 + }, + { + "id": "10902", + "score": 2.7623348535297314 + }, + { + "id": "10897", + "score": 2.7623348535297314 + }, + { + "id": "10892", + "score": 2.7623348535297314 + }, + { + "id": "10887", + "score": 2.7623348535297314 + }, + { + "id": "10882", + "score": 2.7623348535297314 + }, + { + "id": "10877", + "score": 2.7623348535297314 + }, + { + "id": "10872", + "score": 2.7623348535297314 + }, + { + "id": "10867", + "score": 2.7623348535297314 + }, + { + "id": "10862", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.27541155237618664, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "север-from", + "category": "from", + "relevant": { + "10605": 2 + }, + "lexical": [ + { + "id": "10605", + "score": 1 + } + ], + "semantic": [ + { + "id": "10605", + "score": 0.8706388952056376, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-chat", + "category": "chat", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10605", + "score": 1 + }, + { + "id": "10606", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10807", + "score": 0.8763211209875281, + "lexicalRank": 21 + }, + { + "id": "10879", + "score": 0.8751973569555798, + "lexicalRank": 47 + }, + { + "id": "10837", + "score": 0.8742873757382752, + "lexicalRank": 15 + }, + { + "id": "10810", + "score": 0.8742405098346124, + "lexicalRank": 101 + }, + { + "id": "10839", + "score": 0.8741222224299756, + "lexicalRank": 55 + }, + { + "id": "10812", + "score": 0.873962415767805, + "lexicalRank": 20 + }, + { + "id": "10899", + "score": 0.8736988678008778, + "lexicalRank": 43 + }, + { + "id": "10865", + "score": 0.8733206746761927, + "lexicalRank": 90 + }, + { + "id": "10859", + "score": 0.8731440824259407, + "lexicalRank": 51 + }, + { + "id": "10842", + "score": 0.8728835335529379, + "lexicalRank": 14 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-date", + "category": "date", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10605", + "score": 1 + }, + { + "id": "10606", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10605", + "score": 0.9013927195872135, + "lexicalRank": 0 + }, + { + "id": "10606", + "score": 0.8751312341989249, + "lexicalRank": 1 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-and", + "category": "and", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "север-or", + "category": "or", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "10606", + "score": 8.47765579013484 + }, + { + "id": "10605", + "score": 8.32129250405688 + }, + { + "id": "10902", + "score": 7.489838832938161 + }, + { + "id": "10897", + "score": 7.489838832938161 + }, + { + "id": "10892", + "score": 7.489838832938161 + }, + { + "id": "10887", + "score": 7.489838832938161 + }, + { + "id": "10882", + "score": 7.489838832938161 + }, + { + "id": "10877", + "score": 7.489838832938161 + }, + { + "id": "10872", + "score": 7.489838832938161 + }, + { + "id": "10867", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "север-paraphrase", + "category": "paraphrase", + "relevant": { + "10605": 2, + "10606": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "север-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-words", + "category": "words", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "11116", + "score": 0.9267224968720181, + "lexicalRank": 140 + }, + { + "id": "11206", + "score": 0.9258146122951406, + "lexicalRank": 122 + }, + { + "id": "10986", + "score": 0.9240344536750787, + "lexicalRank": 166 + }, + { + "id": "10981", + "score": 0.9221301442533495, + "lexicalRank": 167 + }, + { + "id": "11181", + "score": 0.9219162390030617, + "lexicalRank": 127 + }, + { + "id": "11176", + "score": 0.9211905930950566, + "lexicalRank": 128 + }, + { + "id": "11146", + "score": 0.9208713061575157, + "lexicalRank": 134 + }, + { + "id": "11066", + "score": 0.9207864465640064, + "lexicalRank": 150 + }, + { + "id": "11026", + "score": 0.9204614961224764, + "lexicalRank": 158 + }, + { + "id": "11051", + "score": 0.9202586147803542, + "lexicalRank": 153 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-typo", + "category": "typo", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "11116", + "score": 0.9165621307019595, + "lexicalRank": 140 + }, + { + "id": "11206", + "score": 0.9156913304301528, + "lexicalRank": 122 + }, + { + "id": "10986", + "score": 0.9138965186233041, + "lexicalRank": 166 + }, + { + "id": "11176", + "score": 0.9113988597400993, + "lexicalRank": 128 + }, + { + "id": "10981", + "score": 0.9112093128911051, + "lexicalRank": 167 + }, + { + "id": "11181", + "score": 0.9108976500245898, + "lexicalRank": 127 + }, + { + "id": "11066", + "score": 0.9106072424561344, + "lexicalRank": 150 + }, + { + "id": "11051", + "score": 0.9105505198871539, + "lexicalRank": 153 + }, + { + "id": "11146", + "score": 0.9104861584190448, + "lexicalRank": 134 + }, + { + "id": "11142", + "score": 0.9102258582004341, + "lexicalRank": 255 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-beginnings", + "category": "beginnings", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "11206", + "score": 0.9166903870568667, + "lexicalRank": 122 + }, + { + "id": "10986", + "score": 0.9141907260910435, + "lexicalRank": 166 + }, + { + "id": "11116", + "score": 0.914190001936052, + "lexicalRank": 140 + }, + { + "id": "11176", + "score": 0.9122303629510391, + "lexicalRank": 128 + }, + { + "id": "11026", + "score": 0.9122086171694861, + "lexicalRank": 158 + }, + { + "id": "10981", + "score": 0.9121628056814256, + "lexicalRank": 167 + }, + { + "id": "11146", + "score": 0.9121281273747303, + "lexicalRank": 134 + }, + { + "id": "11066", + "score": 0.9121046576597046, + "lexicalRank": 150 + }, + { + "id": "11031", + "score": 0.9119792966738572, + "lexicalRank": 157 + }, + { + "id": "11076", + "score": 0.9116270432881954, + "lexicalRank": 148 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-phrase", + "category": "phrase", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 2.7623348535297314 + }, + { + "id": "11199", + "score": 2.7623348535297314 + }, + { + "id": "11194", + "score": 2.7623348535297314 + }, + { + "id": "11189", + "score": 2.7623348535297314 + }, + { + "id": "11184", + "score": 2.7623348535297314 + }, + { + "id": "11179", + "score": 2.7623348535297314 + }, + { + "id": "11174", + "score": 2.7623348535297314 + }, + { + "id": "11169", + "score": 2.7623348535297314 + }, + { + "id": "11164", + "score": 2.7623348535297314 + }, + { + "id": "11159", + "score": 2.7623348535297314 + } + ], + "semantic": [ + { + "id": "11204", + "score": 2.7623348535297314 + }, + { + "id": "11199", + "score": 2.7623348535297314 + }, + { + "id": "11194", + "score": 2.7623348535297314 + }, + { + "id": "11189", + "score": 2.7623348535297314 + }, + { + "id": "11184", + "score": 2.7623348535297314 + }, + { + "id": "11179", + "score": 2.7623348535297314 + }, + { + "id": "11174", + "score": 2.7623348535297314 + }, + { + "id": "11169", + "score": 2.7623348535297314 + }, + { + "id": "11164", + "score": 2.7623348535297314 + }, + { + "id": "11159", + "score": 2.7623348535297314 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "lumen-from", + "category": "from", + "relevant": { + "10907": 2 + }, + "lexical": [ + { + "id": "10907", + "score": 1 + } + ], + "semantic": [ + { + "id": "10907", + "score": 0.8844509482138534, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-chat", + "category": "chat", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ], + "semantic": [ + { + "id": "10907", + "score": 0.8739870400246, + "lexicalRank": 21 + }, + { + "id": "11149", + "score": 0.8672188484708652, + "lexicalRank": 11 + }, + { + "id": "11181", + "score": 0.8671707718298383, + "lexicalRank": 47 + }, + { + "id": "11201", + "score": 0.8661931762492886, + "lexicalRank": 43 + }, + { + "id": "11146", + "score": 0.8652178390788425, + "lexicalRank": 54 + }, + { + "id": "11156", + "score": 0.8651391606948476, + "lexicalRank": 52 + }, + { + "id": "11176", + "score": 0.8649251345654683, + "lexicalRank": 48 + }, + { + "id": "11206", + "score": 0.8648163875420569, + "lexicalRank": 42 + }, + { + "id": "11196", + "score": 0.8644069415700817, + "lexicalRank": 44 + }, + { + "id": "11151", + "score": 0.8642576487131611, + "lexicalRank": 53 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0.5, + "mrr": 1, + "ndcgAt10": 0.8262346571285599, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-date", + "category": "date", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "10908", + "score": 1 + }, + { + "id": "10907", + "score": 0.9838709677419355 + } + ], + "semantic": [ + { + "id": "10907", + "score": 0.8694133177629458, + "lexicalRank": 1 + }, + { + "id": "10908", + "score": 0.8560303724815496, + "lexicalRank": 0 + } + ], + "lexicalMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 1, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-and", + "category": "and", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "lumen-or", + "category": "or", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "semantic": [ + { + "id": "11204", + "score": 7.489838832938161 + }, + { + "id": "11199", + "score": 7.489838832938161 + }, + { + "id": "11194", + "score": 7.489838832938161 + }, + { + "id": "11189", + "score": 7.489838832938161 + }, + { + "id": "11184", + "score": 7.489838832938161 + }, + { + "id": "11179", + "score": 7.489838832938161 + }, + { + "id": "11174", + "score": 7.489838832938161 + }, + { + "id": "11169", + "score": 7.489838832938161 + }, + { + "id": "11164", + "score": 7.489838832938161 + }, + { + "id": "11159", + "score": 7.489838832938161 + } + ], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": false + }, + { + "id": "lumen-paraphrase", + "category": "paraphrase", + "relevant": { + "10907": 2, + "10908": 1 + }, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-absent", + "category": "no-answer-absent", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + }, + { + "id": "lumen-near-miss", + "category": "no-answer-near-miss", + "relevant": {}, + "lexical": [], + "semantic": [], + "lexicalMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "semanticMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "expanded": true + } + ], + "questionRows": [ + { + "id": "aurora-question-decision", + "category": "decision", + "relevant": { + "10001": 2, + "10002": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10177", + "score": 0.8743954334334848, + "lexicalRank": 207 + }, + { + "id": "10125", + "score": 0.8742033104489153, + "lexicalRank": 157 + }, + { + "id": "10065", + "score": 0.8740301411239777, + "lexicalRank": 169 + }, + { + "id": "10085", + "score": 0.8736044415926192, + "lexicalRank": 165 + }, + { + "id": "10282", + "score": 0.8734653875377917, + "lexicalRank": 186 + }, + { + "id": "10165", + "score": 0.8732298685541525, + "lexicalRank": 149 + }, + { + "id": "10127", + "score": 0.8730335824366351, + "lexicalRank": 217 + }, + { + "id": "10257", + "score": 0.8727338156186585, + "lexicalRank": 191 + }, + { + "id": "10152", + "score": 0.8727318267187806, + "lexicalRank": 212 + }, + { + "id": "10197", + "score": 0.8725556148157968, + "lexicalRank": 203 + } + ], + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "aurora-question-question", + "category": "question", + "relevant": { + "10001": 2, + "10002": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10279", + "score": 0.9055505730054375, + "lexicalRank": 65 + }, + { + "id": "10259", + "score": 0.9053856622231062, + "lexicalRank": 69 + }, + { + "id": "10189", + "score": 0.9053339695175756, + "lexicalRank": 83 + }, + { + "id": "10239", + "score": 0.9053000598760494, + "lexicalRank": 73 + }, + { + "id": "10199", + "score": 0.9049139536790939, + "lexicalRank": 81 + }, + { + "id": "10164", + "score": 0.9016095617494763, + "lexicalRank": 88 + }, + { + "id": "10204", + "score": 0.9013706164104074, + "lexicalRank": 80 + }, + { + "id": "10214", + "score": 0.9008109120007896, + "lexicalRank": 78 + }, + { + "id": "10294", + "score": 0.9007152019365965, + "lexicalRank": 62 + }, + { + "id": "10224", + "score": 0.9005188968223997, + "lexicalRank": 76 + } + ], + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "aurora-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10165", + "score": 0.8726802210445395, + "lexicalRank": 149 + }, + { + "id": "10085", + "score": 0.8709543606664426, + "lexicalRank": 165 + }, + { + "id": "10130", + "score": 0.8706648612232869, + "lexicalRank": 156 + }, + { + "id": "10065", + "score": 0.8703101048025961, + "lexicalRank": 169 + }, + { + "id": "10110", + "score": 0.86957869880922, + "lexicalRank": 160 + }, + { + "id": "10095", + "score": 0.8694762124215225, + "lexicalRank": 163 + }, + { + "id": "10245", + "score": 0.8693494230668928, + "lexicalRank": 133 + }, + { + "id": "10125", + "score": 0.8691772982774707, + "lexicalRank": 157 + }, + { + "id": "10190", + "score": 0.8691229625304249, + "lexicalRank": 144 + }, + { + "id": "10270", + "score": 0.8690160101262605, + "lexicalRank": 128 + } + ], + "lexical": [ + { + "id": "10298", + "score": 1 + }, + { + "id": "10293", + "score": 0.9838709677419355 + }, + { + "id": "10288", + "score": 0.9682539682539681 + }, + { + "id": "10283", + "score": 0.953125 + }, + { + "id": "10278", + "score": 0.9384615384615385 + }, + { + "id": "10273", + "score": 0.9242424242424243 + }, + { + "id": "10268", + "score": 0.9104477611940298 + }, + { + "id": "10263", + "score": 0.8970588235294118 + }, + { + "id": "10258", + "score": 0.8840579710144928 + }, + { + "id": "10253", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "cedar-question-decision", + "category": "decision", + "relevant": { + "10303": 2, + "10304": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10407", + "score": 0.8921191588753772, + "lexicalRank": 161 + }, + { + "id": "10457", + "score": 0.8913712287362181, + "lexicalRank": 151 + }, + { + "id": "10582", + "score": 0.8912151177976313, + "lexicalRank": 126 + }, + { + "id": "10412", + "score": 0.8909905149008674, + "lexicalRank": 160 + }, + { + "id": "10367", + "score": 0.8901243868500228, + "lexicalRank": 169 + }, + { + "id": "10342", + "score": 0.889620914359146, + "lexicalRank": 174 + }, + { + "id": "10482", + "score": 0.8895100338046731, + "lexicalRank": 146 + }, + { + "id": "10512", + "score": 0.8884118601789093, + "lexicalRank": 140 + }, + { + "id": "10427", + "score": 0.8876010515238454, + "lexicalRank": 157 + }, + { + "id": "10357", + "score": 0.8873435565112464, + "lexicalRank": 171 + } + ], + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "cedar-question-question", + "category": "question", + "relevant": { + "10303": 2, + "10304": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10303", + "score": 0.8997394063896005, + "lexicalRank": 60 + }, + { + "id": "10412", + "score": 0.870363248650828, + "lexicalRank": 160 + }, + { + "id": "10457", + "score": 0.8684160915373113, + "lexicalRank": 151 + }, + { + "id": "10407", + "score": 0.8673949648596698, + "lexicalRank": 161 + }, + { + "id": "10367", + "score": 0.8665641081528737, + "lexicalRank": 169 + }, + { + "id": "10431", + "score": 0.8658399495887587, + "lexicalRank": 95 + }, + { + "id": "10526", + "score": 0.8655811438748423, + "lexicalRank": 76 + }, + { + "id": "10471", + "score": 0.8647088527163593, + "lexicalRank": 87 + }, + { + "id": "10506", + "score": 0.8642371766217116, + "lexicalRank": 80 + }, + { + "id": "10447", + "score": 0.8640630498264111, + "lexicalRank": 153 + } + ], + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "cedar-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10407", + "score": 0.8699333934870884, + "lexicalRank": 161 + }, + { + "id": "10357", + "score": 0.8687598750179746, + "lexicalRank": 171 + }, + { + "id": "10367", + "score": 0.8683958480929811, + "lexicalRank": 169 + }, + { + "id": "10412", + "score": 0.8681930254871126, + "lexicalRank": 160 + }, + { + "id": "10457", + "score": 0.8681691135946581, + "lexicalRank": 151 + }, + { + "id": "10482", + "score": 0.8681629432596382, + "lexicalRank": 146 + }, + { + "id": "10602", + "score": 0.8675038158306415, + "lexicalRank": 122 + }, + { + "id": "10362", + "score": 0.8671538232633393, + "lexicalRank": 170 + }, + { + "id": "10342", + "score": 0.8666654261657587, + "lexicalRank": 174 + }, + { + "id": "10402", + "score": 0.8659104858701784, + "lexicalRank": 162 + } + ], + "lexical": [ + { + "id": "10600", + "score": 1 + }, + { + "id": "10595", + "score": 0.9838709677419355 + }, + { + "id": "10590", + "score": 0.9682539682539681 + }, + { + "id": "10585", + "score": 0.953125 + }, + { + "id": "10580", + "score": 0.9384615384615385 + }, + { + "id": "10575", + "score": 0.9242424242424243 + }, + { + "id": "10570", + "score": 0.9104477611940298 + }, + { + "id": "10565", + "score": 0.8970588235294118 + }, + { + "id": "10560", + "score": 0.8840579710144928 + }, + { + "id": "10555", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "север-question-decision", + "category": "decision", + "relevant": { + "10605": 2, + "10606": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10605", + "score": 0.8900251732962743, + "lexicalRank": 1 + }, + { + "id": "10879", + "score": 0.8735637560863058, + "lexicalRank": 127 + }, + { + "id": "10744", + "score": 0.8729735818123653, + "lexicalRank": 154 + }, + { + "id": "10606", + "score": 0.8727974639516136, + "lexicalRank": 0 + }, + { + "id": "10789", + "score": 0.8724961516351566, + "lexicalRank": 145 + }, + { + "id": "10854", + "score": 0.8721360361721472, + "lexicalRank": 132 + }, + { + "id": "10844", + "score": 0.8721247099894074, + "lexicalRank": 134 + }, + { + "id": "10659", + "score": 0.8716292153677793, + "lexicalRank": 171 + }, + { + "id": "10799", + "score": 0.871430651766806, + "lexicalRank": 143 + }, + { + "id": "10849", + "score": 0.8712351264135738, + "lexicalRank": 133 + } + ], + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "север-question-question", + "category": "question", + "relevant": { + "10605": 2, + "10606": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 1, + "mrr": 1, + "ndcgAt10": 0.7967075809905066, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10605", + "score": 0.891559997434239, + "lexicalRank": 1 + }, + { + "id": "10606", + "score": 0.8685433570269113, + "lexicalRank": 0 + }, + { + "id": "10744", + "score": 0.8646542190680107, + "lexicalRank": 154 + }, + { + "id": "10879", + "score": 0.8640846188885009, + "lexicalRank": 127 + }, + { + "id": "10660", + "score": 0.864000293770232, + "lexicalRank": 291 + }, + { + "id": "10799", + "score": 0.8633917953590933, + "lexicalRank": 143 + }, + { + "id": "10854", + "score": 0.8628477048736447, + "lexicalRank": 132 + }, + { + "id": "10844", + "score": 0.8628242883654791, + "lexicalRank": 134 + }, + { + "id": "10720", + "score": 0.8625328190572694, + "lexicalRank": 279 + }, + { + "id": "10650", + "score": 0.8618473253582783, + "lexicalRank": 293 + } + ], + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "север-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10606", + "score": 0.8688934670502048, + "lexicalRank": 0 + }, + { + "id": "10605", + "score": 0.8630904012969371, + "lexicalRank": 1 + }, + { + "id": "10879", + "score": 0.8544312398339066, + "lexicalRank": 127 + }, + { + "id": "10854", + "score": 0.852517259859595, + "lexicalRank": 132 + }, + { + "id": "10744", + "score": 0.8518251851948647, + "lexicalRank": 154 + }, + { + "id": "10639", + "score": 0.8497193354703592, + "lexicalRank": 175 + }, + { + "id": "10899", + "score": 0.8495133421303337, + "lexicalRank": 123 + }, + { + "id": "10789", + "score": 0.8494706401084311, + "lexicalRank": 145 + }, + { + "id": "10889", + "score": 0.8493093202619589, + "lexicalRank": 125 + }, + { + "id": "10849", + "score": 0.8492858369106064, + "lexicalRank": 133 + } + ], + "lexical": [ + { + "id": "10606", + "score": 1 + }, + { + "id": "10605", + "score": 0.9838709677419355 + }, + { + "id": "10902", + "score": 0.9682539682539681 + }, + { + "id": "10897", + "score": 0.953125 + }, + { + "id": "10892", + "score": 0.9384615384615385 + }, + { + "id": "10887", + "score": 0.9242424242424243 + }, + { + "id": "10882", + "score": 0.9104477611940298 + }, + { + "id": "10877", + "score": 0.8970588235294118 + }, + { + "id": "10872", + "score": 0.8840579710144928 + }, + { + "id": "10867", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "lumen-question-decision", + "category": "decision", + "relevant": { + "10907": 2, + "10908": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10907", + "score": 0.8894495730865635, + "lexicalRank": 61 + }, + { + "id": "10908", + "score": 0.8708438836496035, + "lexicalRank": 60 + }, + { + "id": "11206", + "score": 0.8608495452142796, + "lexicalRank": 122 + }, + { + "id": "11010", + "score": 0.8576054454020773, + "lexicalRank": 101 + }, + { + "id": "10986", + "score": 0.8572175978877122, + "lexicalRank": 166 + }, + { + "id": "11146", + "score": 0.8568726575802139, + "lexicalRank": 134 + }, + { + "id": "11165", + "score": 0.8561342933704561, + "lexicalRank": 70 + }, + { + "id": "11116", + "score": 0.8557647490617573, + "lexicalRank": 140 + }, + { + "id": "11080", + "score": 0.8555912262027077, + "lexicalRank": 87 + }, + { + "id": "11066", + "score": 0.8553978516789799, + "lexicalRank": 150 + } + ], + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "lumen-question-question", + "category": "question", + "relevant": { + "10907": 2, + "10908": 1 + }, + "directMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": 0, + "mrr": 0, + "ndcgAt10": 0, + "falseHits": 0 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "10907", + "score": 0.8786636894499708, + "lexicalRank": 61 + }, + { + "id": "10908", + "score": 0.8569269701977336, + "lexicalRank": 60 + }, + { + "id": "11206", + "score": 0.8514216173795147, + "lexicalRank": 122 + }, + { + "id": "11140", + "score": 0.8503204493298434, + "lexicalRank": 75 + }, + { + "id": "11146", + "score": 0.8496648530020674, + "lexicalRank": 134 + }, + { + "id": "11010", + "score": 0.8495310712219786, + "lexicalRank": 101 + }, + { + "id": "11120", + "score": 0.8485951608616598, + "lexicalRank": 79 + }, + { + "id": "11105", + "score": 0.8477906106928664, + "lexicalRank": 82 + }, + { + "id": "11065", + "score": 0.8472162620563428, + "lexicalRank": 90 + }, + { + "id": "11116", + "score": 0.8470660011285048, + "lexicalRank": 140 + } + ], + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ] + }, + { + "id": "lumen-question-no-answer", + "category": "no-answer", + "relevant": {}, + "directMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 0 + }, + "anchorMetrics": { + "recallAt10": null, + "mrr": null, + "ndcgAt10": null, + "falseHits": 10 + }, + "directCandidateCount": 0, + "anchoredCandidateCount": 300, + "semantic": [ + { + "id": "11206", + "score": 0.8733068842004856, + "lexicalRank": 122 + }, + { + "id": "11146", + "score": 0.871852601191307, + "lexicalRank": 134 + }, + { + "id": "11066", + "score": 0.8713609653818737, + "lexicalRank": 150 + }, + { + "id": "11181", + "score": 0.8705610095113754, + "lexicalRank": 127 + }, + { + "id": "11006", + "score": 0.8692922844813956, + "lexicalRank": 162 + }, + { + "id": "11116", + "score": 0.8692283014040969, + "lexicalRank": 140 + }, + { + "id": "10981", + "score": 0.8691945640873089, + "lexicalRank": 167 + }, + { + "id": "11076", + "score": 0.8687983001696971, + "lexicalRank": 148 + }, + { + "id": "11126", + "score": 0.8687221972857682, + "lexicalRank": 138 + }, + { + "id": "11026", + "score": 0.8683696685646625, + "lexicalRank": 158 + } + ], + "lexical": [ + { + "id": "11204", + "score": 1 + }, + { + "id": "11199", + "score": 0.9838709677419355 + }, + { + "id": "11194", + "score": 0.9682539682539681 + }, + { + "id": "11189", + "score": 0.953125 + }, + { + "id": "11184", + "score": 0.9384615384615385 + }, + { + "id": "11179", + "score": 0.9242424242424243 + }, + { + "id": "11174", + "score": 0.9104477611940298 + }, + { + "id": "11169", + "score": 0.8970588235294118 + }, + { + "id": "11164", + "score": 0.8840579710144928 + }, + { + "id": "11159", + "score": 0.8714285714285714 + } + ] + } + ] +} diff --git a/bench/message-search-quality/semantic.ts b/bench/message-search-quality/semantic.ts new file mode 100644 index 000000000..54b8ce8cf --- /dev/null +++ b/bench/message-search-quality/semantic.ts @@ -0,0 +1,422 @@ +import assert from "node:assert/strict" +import { createHash } from "node:crypto" +import { mkdtempSync, readFileSync, writeFileSync } from "node:fs" +import { cpus, tmpdir } from "node:os" +import { join } from "node:path" +import { performance } from "node:perf_hooks" +import type { Message } from "../../dist/domain/models.js" +import { isTextModelInstalled, openEmbedder, textModelsDirectory } from "../../dist/embeddings/embed.js" +import { textModel } from "../../dist/embeddings/models.js" +import { searchCombined } from "../../dist/services/messages-combined.js" +import { openStore } from "../../dist/store/store.js" +import { openCrossEncoder } from "./crossencoder.ts" + +type Query = { id: string; split: "dev" | "test"; category: string; lucene: string; relevant: Record } +type Question = { + id: string + split: "dev" | "test" + category: string + retrieval: string + text: string + relevant: Record +} +type Seed = { + chats: { id: string; title: string }[] + messages: Pick[] +} +type Ranking = { id: string; score: number } +type Metrics = { recallAt10: number | null; mrr: number | null; ndcgAt10: number | null; falseHits: number } +const digest = (bytes: Buffer) => createHash("sha256").update(bytes).digest("hex") +const originalBytes = readFileSync(new URL("./corpus.json", import.meta.url)) +const questionBytes = readFileSync(new URL("./questions.json", import.meta.url)) +const corpus = JSON.parse(originalBytes.toString()) as Seed & { queries: Query[] } +const supplement = JSON.parse(questionBytes.toString()) as Seed & { queries: Question[] } +const root = mkdtempSync(join(tmpdir(), "semantic-message-quality-")) +const crossEncoderMode = process.argv.includes("--crossencoder") +const floors: (number | null)[] = crossEncoderMode + ? [null, -5, -2, -1, 0, 1, 2, 3, 5, 8] + : [null, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95] +const crossEncoderDirectory = process.env.MESSAGE_RERANKER_DIR +if (crossEncoderMode) + assert.ok(crossEncoderDirectory, "MESSAGE_RERANKER_DIR must name a separately prepared local model") +const directory = textModelsDirectory() +const model = textModel("e5-small") +if (!crossEncoderMode) + assert.ok( + isTextModelInstalled(model, directory), + "Install the pinned model separately before running; this benchmark never downloads it", + ) +if (!crossEncoderMode) + for (const file of model.files) + assert.equal(digest(readFileSync(join(directory, model.id, file.name))), file.sha256, `pinned model ${file.name}`) +process.env.TZ = "UTC" +process.env.MESSAGING_STORE = join(root, "quality.db") +process.env.CLI_COMMON_CACHE_DIR = join(root, "empty-cache") +const store = await openStore({ path: process.env.MESSAGING_STORE }) +const account = { provider: "synthetic", account: "500" } +const depths = [50, 100, 300, 500] +const seeds = [corpus, supplement] +const allMessages = seeds.flatMap((seed) => seed.messages) +const messages = new Map(allMessages.map((m) => [m.id, m])) +const vectorCache = new Map() +const queryCache = new Map() +const modeFiles = [ + "services/messages-combined.js", + "search/lucene/parser.js", + "embeddings/embed.js", + "embeddings/models.js", + "store/sqlite/lucene.js", +] +const frozenInputs = { + crossEncoderImplementationSha256: digest(readFileSync(new URL("./crossencoder.ts", import.meta.url))), + rerankerManifestSha256: digest(readFileSync(new URL("./reranker.json", import.meta.url))), + corpusSha256: digest(originalBytes), + questionsSha256: digest(questionBytes), + runnerSha256: digest(readFileSync(new URL("./semantic.ts", import.meta.url))), + productionSha256: Object.fromEntries( + modeFiles.map((file) => [file, digest(readFileSync(new URL(`../../dist/${file}`, import.meta.url)))]), + ), +} +writeFileSync(join(root, "frozen-inputs.json"), JSON.stringify(frozenInputs, null, 2)) +const metrics = (rankings: Ranking[], relevant: Record): Metrics => { + const ids = rankings.slice(0, 10).map((h) => h.id) + const grades = Object.values(relevant) + if (!grades.length) return { recallAt10: null, mrr: null, ndcgAt10: null, falseHits: ids.length } + const first = ids.findIndex((id) => relevant[id]) + const dcg = (g: number[]) => g.reduce((s, v, i) => s + (2 ** v - 1) / Math.log2(i + 2), 0) + return { + recallAt10: ids.filter((id) => relevant[id]).length / grades.length, + mrr: first < 0 ? 0 : 1 / (first + 1), + ndcgAt10: dcg(ids.map((id) => relevant[id] ?? 0)) / dcg([...grades].sort((a, b) => b - a).slice(0, 10)), + falseHits: 0, + } +} +const summarize = (rows: Metrics[]) => { + const answered = rows.filter((row) => row.recallAt10 !== null) + const avg = (key: "recallAt10" | "mrr" | "ndcgAt10") => + answered.length ? answered.reduce((s, r) => s + (r[key] ?? 0), 0) / answered.length : 0 + return { + answerable: answered.length, + noAnswerQueries: rows.length - answered.length, + recallAt10: avg("recallAt10"), + mrr: avg("mrr"), + ndcgAt10: avg("ndcgAt10"), + noAnswerFalseHits: rows.reduce((s, r) => s + r.falseHits, 0), + } +} +const options = (depth: number) => ({ candidateDepth: depth, rrfK: 60, rerank: "none" as const, chatCap: 0 }) +const pool = async (text: string, depth: number) => { + const found = await searchCombined(store, account, { text, limit: depth, timezone: "UTC" }, {}, options(depth)) + assert.equal(new Set(found.items.map((h) => h.id)).size, found.items.length) + assert.ok(found.items.every((h) => messages.has(h.id))) + return { + hits: found.items.map((h) => ({ id: h.id, score: h.score ?? 0 })), + expanded: found.query.combined.expanded, + truncated: found.hasMore, + } +} +const cosine = (a: Float32Array, b: Float32Array) => { + assert.equal(a.length, b.length) + let score = 0 + for (let i = 0; i < a.length; i++) score += (a[i] ?? 0) * (b[i] ?? 0) + return score +} +let embedder: Awaited> | undefined +let crossEncoder: Awaited> | undefined +const pairCache = new Map() +let pairInferenceMs = 0, + scoredPairs = 0 +let passageInferenceMs = 0, + queryInferenceMs = 0, + embeddedMessages = 0 +const rerank = async (text: string, candidates: Ranking[]) => { + if (crossEncoder) { + const ranked = [] + const start = performance.now() + for (const [lexicalRank, hit] of candidates.entries()) { + const key = JSON.stringify([text, hit.id]) + let score = pairCache.get(key) + if (score === undefined) { + assert.ok(scoredPairs < 20000, "bounded experimental pair budget") + score = await crossEncoder.score(text, messages.get(hit.id)?.text ?? "") + pairCache.set(key, score) + scoredPairs++ + } + ranked.push({ id: hit.id, score, lexicalRank }) + } + pairInferenceMs += performance.now() - start + console.error(`Scored ${scoredPairs} synthetic query-message pairs`) + return ranked.sort((a, b) => b.score - a.score || a.lexicalRank - b.lexicalRank) + } + assert.ok(embedder) + let query = queryCache.get(text) + if (!query) { + const start = performance.now() + const [vector] = await embedder.embed([text], "query") + assert.ok(vector) + query = vector + queryCache.set(text, query) + queryInferenceMs += performance.now() - start + } + const missing = candidates.filter((hit) => !vectorCache.has(hit.id)) + if (missing.length) { + const start = performance.now() + const vectors = await embedder.embed( + missing.map((hit) => messages.get(hit.id)?.text ?? ""), + "passage", + ) + assert.equal(vectors.length, missing.length) + missing.forEach((hit, i) => { + const vector = vectors[i] + assert.ok(vector) + vectorCache.set(hit.id, vector) + }) + passageInferenceMs += performance.now() - start + embeddedMessages += missing.length + console.error(`Embedded ${embeddedMessages} synthetic candidates; no network or model download`) + } + return candidates + .map((hit, rank) => { + const vector = vectorCache.get(hit.id) + assert.ok(vector) + return { id: hit.id, score: cosine(query, vector), lexicalRank: rank } + }) + .sort((a, b) => b.score - a.score || a.lexicalRank - b.lexicalRank) +} +try { + for (const seed of seeds) { + await store.saveChats( + account, + seed.chats.map((c) => ({ ...c, kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 100 })), + ) + for (const chat of seed.chats) + await store.saveMessages( + account, + chat.id, + seed.messages + .filter((m) => m.chatId === chat.id) + .map( + (m): Message => ({ + ...m, + senderName: `Synthetic ${m.senderId}`, + editedAt: null, + outgoing: false, + attachments: [], + replyTo: null, + forwardedFrom: null, + reactions: null, + }), + ), + { via: "synthetic" }, + ) + } + await store.fillSearchIndex({}) + await store.fillStems({}) + const dev = corpus.queries.filter((q) => q.split === "dev") + const candidateRows: (Metrics & { + id: string + category: string + depth: number + relevant: Record + candidateIds: string[] + candidateRecall: number | null + truncated: boolean + })[] = [] + for (const q of dev) { + for (const depth of depths) { + const candidates = await pool(q.lucene, depth) + const labels = Object.keys(q.relevant) + candidateRows.push({ + id: q.id, + category: q.category, + depth, + relevant: q.relevant, + candidateIds: candidates.hits.map((h) => h.id), + candidateRecall: labels.length + ? labels.filter((id) => candidates.hits.some((h) => h.id === id)).length / labels.length + : null, + ...metrics(candidates.hits, q.relevant), + truncated: candidates.truncated, + }) + } + } + const candidateSummary = depths.map((depth) => { + const rows = candidateRows.filter((r) => r.depth === depth && r.candidateRecall !== null) + return { + depth, + candidateRecall: rows.reduce((s, r) => s + (r.candidateRecall ?? 0), 0) / rows.length, + byCategory: Object.fromEntries( + [...new Set(rows.map((r) => r.category))].map((category) => { + const selected = rows.filter((r) => r.category === category) + return [category, selected.reduce((s, r) => s + (r.candidateRecall ?? 0), 0) / selected.length] + }), + ), + } + }) + const maxCandidateRecall = Math.max(...candidateSummary.map((r) => r.candidateRecall)) + const selectedDepth = candidateSummary.find((r) => r.candidateRecall === maxCandidateRecall)?.depth ?? 500 + console.error(`Dev candidate recall measured; selected depth ${selectedDepth} on dev only`) + const loaded = performance.now() + if (crossEncoderMode) { + assert.ok(crossEncoderDirectory) + crossEncoder = await openCrossEncoder(crossEncoderDirectory) + } else embedder = await openEmbedder(model, directory, { threads: 8 }) + const loadMs = performance.now() - loaded + const keywordRows = [] + for (const q of dev) { + const candidates = await pool(q.lucene, selectedDepth) + const semantic = candidates.expanded ? await rerank(q.lucene, candidates.hits) : candidates.hits + keywordRows.push({ + id: q.id, + category: q.category, + relevant: q.relevant, + lexical: candidates.hits.slice(0, 10), + semantic: semantic.slice(0, 10), + lexicalMetrics: metrics(candidates.hits, q.relevant), + semanticMetrics: metrics(semantic, q.relevant), + expanded: candidates.expanded, + }) + } + const questionRows: { + id: string + category: string + relevant: Record + directMetrics: Metrics + anchorMetrics: Metrics + directCandidateCount: number + anchoredCandidateCount: number + semantic: Ranking[] + lexical: Ranking[] + }[] = [] + for (const q of supplement.queries.filter((q) => q.split === "dev")) { + const direct = await pool(q.text, selectedDepth) + const anchored = await pool(q.retrieval, selectedDepth) + const semantic = await rerank(q.text, anchored.hits) + questionRows.push({ + id: q.id, + category: q.category, + relevant: q.relevant, + directMetrics: metrics(direct.hits, q.relevant), + anchorMetrics: metrics(anchored.hits, q.relevant), + directCandidateCount: direct.hits.length, + anchoredCandidateCount: anchored.hits.length, + semantic, + lexical: anchored.hits.slice(0, 10), + }) + } + const keywordSummary = { + lexical: summarize(keywordRows.map((r) => r.lexicalMetrics)), + semantic: summarize(keywordRows.map((r) => r.semanticMetrics)), + } + const questionSweep = floors.map((floor) => ({ + floor, + metrics: summarize( + questionRows.map((r) => + metrics( + r.semantic.filter((h) => floor === null || h.score > floor), + r.relevant, + ), + ), + ), + })) + const passing = (m: ReturnType) => + m.recallAt10 >= 0.45 && m.mrr >= 0.5 && m.ndcgAt10 >= 0.4 && m.noAnswerFalseHits === 0 + const eligible = questionSweep.filter((row) => passing(row.metrics)) + const chosen = [...(eligible.length ? eligible : questionSweep)].sort( + (a, b) => a.metrics.noAnswerFalseHits - b.metrics.noAnswerFalseHits || b.metrics.ndcgAt10 - a.metrics.ndcgAt10, + )[0] + assert.ok(chosen) + const devQualified = passing(keywordSummary.semantic) && eligible.length > 0 + const heldOutKeywordRows: { + id: string + metrics: Metrics + lexicalMetrics: Metrics + relevant: Record + hits: Ranking[] + }[] = [] + const heldOutRows = [] + if (devQualified) { + for (const q of corpus.queries.filter((q) => q.split === "test")) { + const candidates = await pool(q.lucene, selectedDepth) + const ranked = candidates.expanded ? await rerank(q.lucene, candidates.hits) : candidates.hits + heldOutKeywordRows.push({ + id: q.id, + relevant: q.relevant, + hits: ranked.slice(0, 10), + metrics: metrics(ranked, q.relevant), + lexicalMetrics: metrics(candidates.hits, q.relevant), + }) + } + for (const q of supplement.queries.filter((q) => q.split === "test")) { + const candidates = await pool(q.retrieval, selectedDepth) + const ranked = await rerank(q.text, candidates.hits) + const hits = ranked.filter((h) => chosen.floor === null || h.score > chosen.floor) + heldOutRows.push({ id: q.id, relevant: q.relevant, hits: hits.slice(0, 10), metrics: metrics(hits, q.relevant) }) + } + } + console.log( + JSON.stringify( + { + ...frozenInputs, + node: process.version, + cpu: cpus()[0]?.model, + storePath: process.env.MESSAGING_STORE, + model: crossEncoder?.model ?? { + id: model.id, + files: model.files.map(({ name, sha256 }) => ({ name, sha256 })), + threads: 8, + }, + kind: crossEncoderMode + ? "cross-encoder raw logit reranking" + : "embedding-cosine reranking; not a cross-encoder", + candidateSummary, + selectedOnDev: { depth: selectedDepth, questionScoreFloor: chosen.floor }, + keywordSummary, + questionSummary: { + direct: summarize(questionRows.map((r) => r.directMetrics)), + anchoredLexical: summarize(questionRows.map((r) => r.anchorMetrics)), + sweep: questionSweep, + }, + devQualified, + questionRetrieval: + "manual topic anchors for diagnostic reranking; full-question direct retrieval is measured separately", + heldOut: devQualified + ? { + keyword: { + lexical: summarize(heldOutKeywordRows.map((r) => r.lexicalMetrics)), + semantic: summarize(heldOutKeywordRows.map((r) => r.metrics)), + rows: heldOutKeywordRows, + }, + questions: { metrics: summarize(heldOutRows.map((r) => r.metrics)), rows: heldOutRows }, + } + : null, + inference: { + pairInferenceMs, + scoredPairs, + averagePairMs: scoredPairs ? pairInferenceMs / scoredPairs : null, + loadMs, + embeddedMessages, + uniqueEmbeddedQueries: queryCache.size, + uniqueCrossEncoderQueries: new Set( + [...pairCache.keys()].map((key) => (JSON.parse(key) as [string, string])[0]), + ).size, + passageInferenceMs, + queryInferenceMs, + amortizedPerMessageMs: embeddedMessages ? passageInferenceMs / embeddedMessages : null, + cache: crossEncoderMode + ? "exact synthetic query-message score cache, empty at run start" + : "new in-memory synthetic message vectors; no persistent production message index", + }, + candidateRows, + keywordRows, + questionRows: questionRows.map((row) => ({ ...row, semantic: row.semantic.slice(0, 10) })), + }, + null, + 2, + ), + ) +} finally { + await crossEncoder?.close() + await embedder?.close() + await store.close() +} diff --git a/bench/message-search-quality/tsconfig.json b/bench/message-search-quality/tsconfig.json index 86ade7582..e3b79c572 100644 --- a/bench/message-search-quality/tsconfig.json +++ b/bench/message-search-quality/tsconfig.json @@ -1,5 +1,13 @@ { "extends": "@leemour/cli-core/tsconfig.base.json", - "compilerOptions": { "noEmit": true, "composite": false, "declaration": false, "declarationMap": false }, - "include": ["*.ts"] + "compilerOptions": { + "noEmit": true, + "composite": false, + "declaration": false, + "declarationMap": false, + "allowImportingTsExtensions": true + }, + "include": [ + "*.ts" + ] } diff --git a/docs/dev/combined-search.md b/docs/dev/combined-search.md index b84860b04..cd1e7ee93 100644 --- a/docs/dev/combined-search.md +++ b/docs/dev/combined-search.md @@ -1,8 +1,9 @@ -# One message search: proposal for approval +# One message search: measured design -Status: the owner approved the lexical experiment. Its dev evaluation failed the approved -quality gates, so CLI/MCP and SDK defaults remain unchanged. The internal matcher and measured -results are available for review; the revised semantic evaluation below needs owner approval. +Status: the owner approved the lexical and semantic experiments. Lexical reranking and e5 cosine +ranking fail the dev quality gates; joint query-message ranking improves the measured results. +CLI/MCP and SDK defaults remain unchanged while latency and the original held-out quality gates +remain unmet. Natural-language question retrieval is a separate diagnostic, not a public feature. ## Evidence and acceptance targets @@ -74,9 +75,9 @@ No 100k-message latency claim is made; release-scale performance work follows a The old baseline remains committed unchanged. The smallest experimental configuration is reported for inspection, not selected as a release recommendation; the internal defaults are provisional. -Proposed next phase, requiring approval: measure a local semantic reranker over the bounded, -filter-valid lexical candidates before changing the public default. First define its model, -installation/readiness behavior and cost budget, without an automatic download or network call. +The approved semantic experiment measures a local reranker over bounded, filter-valid lexical +candidates before changing the public default. Model preparation is an explicit separate step; +the benchmark itself cannot download a model, connect to a messenger or open the real store. Evaluate per-message text against the whole query; compare it with chunk-to-message semantic retrieval only if lexical candidate recall itself is inadequate. Do not reuse the conversation 0.80 cosine floor as a message threshold. @@ -87,7 +88,49 @@ this keyword-query benchmark or relabel distractors. The existing `intent` field answer text and must never be passed to retrieval or a reranker. Create fresh held-out topic/query families before tuning the new approach, freeze the dev choice, then evaluate held-out once. If semantic ranking also fails, report that result rather than relax the release gates implicitly. -All CLI/MCP removal, SDK/saved conversion and consumer releases below remain pending quality gates. +All CLI/MCP removal, SDK/saved conversion and consumer releases below remain pending the full +quality, latency and query-contract checks. + +## Semantic experiment + +The [semantic benchmark](../../bench/message-search-quality/README.md#candidate-recall-and-semantic-ranking) +preserves the original corpus/labels and adds separately authored question queries plus fresh +held-out families. Candidate recall reaches 0.90 at depth 300: every non-paraphrase category has +full candidate recall, while paraphrase requests have none. Most misses in the top ten are ranking +errors, not absent candidates. No broader semantic retrieval has been introduced. + +The installed pinned e5-small embedding model reduces dev keyword recall@10 to 0.275 and fails +the quality gates. Clearer questions help it rank some answers, but cosine floors cannot retain +answers while eliminating the new no-answer hits. Reject embedding cosine as this message reranker. + +The benchmark then evaluates an official Apache-2.0 quantized +[query-message model](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1), pinned separately +in [reranker.json](../../bench/message-search-quality/reranker.json). It scores query and message +together rather than comparing separately generated vectors. The benchmark uses its raw identity +logits, with no automatic model installation, and freezes the dev-selected configuration before +held-out inference. This is a benchmark adapter, not a public model catalogue or service addition. + +Question reranking uses explicitly authored topic anchors for candidate retrieval. Sending a full +question directly to the current implicit-conjunction Lucene search returns no candidates. Thus +successful question reranking is diagnostic evidence, not a completed natural-language search path. +The unknown-auditor queries test whether a model can reject a topical message that lacks the +requested fact. The original keyword/no-answer queries remain unchanged and are reported separately. + +The joint reranker improves original dev keyword recall/MRR/nDCG to 0.525/0.553/0.477, with zero +no-answer hits. The frozen original held-out result is 0.438/0.488/0.405, also with zero false hits: +it beats strict and legacy on all three quality measures but misses the 0.45 recall and 0.50 MRR +gates. Do not retune on these held-out results. The same-archive strict/legacy baseline confirms +that supplemental messages do not account for the improvement. The fresh question diagnostic +reaches 0.688 recall, MRR 1.0 and nDCG 0.866, with zero no-answer hits, under manual topic anchors. +Raw results and reproducibility instructions are in the benchmark README. These are small, +synthetic evaluations with shared templates, not evidence of general real-archive performance. + +The measured CPU pair cost is roughly 10–12 ms per message; scoring 300 fresh candidates takes seconds +and misses the proposed 250 ms latency gate. A model load also costs roughly one second. Synthetic +exact-pair caches do not establish production warm latency for a different query. Public adoption +needs the original held-out quality gates and a measured cost reduction, or an explicitly revised +release policy. Natural-language question support additionally needs automatic candidate selection; +the existing public contract remains Lucene expressions. Ranking gains alone do not qualify a release. ## Query behavior From 7e3d933a0330288eb8e1b037fbe629e28a211534 Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 17:36:05 +0200 Subject: [PATCH 05/14] docs(search): explain evaluation metrics and reranking costs --- bench/message-search-quality/README.md | 9 ++ docs/dev/combined-search-evaluation.md | 157 +++++++++++++++++++++++++ docs/dev/combined-search.md | 3 + 3 files changed, 169 insertions(+) create mode 100644 docs/dev/combined-search-evaluation.md diff --git a/bench/message-search-quality/README.md b/bench/message-search-quality/README.md index 136da54df..9072fd74f 100644 --- a/bench/message-search-quality/README.md +++ b/bench/message-search-quality/README.md @@ -191,3 +191,12 @@ roughly a second to load. Scoring 300 uncached pairs therefore costs about 3–4 the proposed 250 ms latency gate. This is measured benchmark inference, not production p95 or an optimized batch/native implementation. Exact-pair caching helps repeated identical requests; it does not remove the cost of a new query. No public service/default or consumer version changed. + +## Interpreting the measurements + +[The evaluation notes](../../docs/dev/combined-search-evaluation.md) explain candidate retrieval, +model scoring, reranking cost and the precise recall/MRR/nDCG denominators. They also record +what each experiment contributed and proposed next metrics. In the original held-out keyword +report, macro recall 0.4375 means an average 43.75 percent of labelled relevant evidence retrieved; +Success@10 is 50 percent, and grade-2 actual-answer Success@10 is 42.5 percent. Those last two +values are descriptive audits of saved rankings, not new model runs or changed release gates. diff --git a/docs/dev/combined-search-evaluation.md b/docs/dev/combined-search-evaluation.md new file mode 100644 index 000000000..c21b9e987 --- /dev/null +++ b/docs/dev/combined-search-evaluation.md @@ -0,0 +1,157 @@ +# Combined message search: evidence, pipeline and measurement + +The public search is unchanged. The internal lexical service and local reranking experiments are +in draft PR [803](https://github.com/leemour/cli-messaging/pull/803). The +[design](combined-search.md) describes the pending transition; the +[benchmark README](../../bench/message-search-quality/README.md) links reproducible inputs, runners, +model pins and raw results. No experiment opens the real store or a messenger account. + +## What the experiments establish + +| Experiment | Evidence | What it contributes | +|---|---|---| +| Strict and legacy baselines | Strict avoids no-answer noise; legacy recovers spelling variants but fills no-answer pages | Separates matching coverage from noise | +| Independent lexical candidates and 180 ranking configurations | Dev recall 0.3625 with no-answer false hits zero; lexical signal variants do not improve the top ten | Retains useful forgiving retrieval without the broad fallback chain; simple lexical signals are insufficient on these fixtures | +| Candidate pool diagnostics | Dev candidate recall 0.4375/0.7500/0.9000/0.9000 at depths 50/100/300/500 | Most labelled messages can be retrieved before ranking; paraphrases are absent from the lexical pool | +| e5-small embedding cosine | Dev recall falls to 0.275; no question threshold both preserves answers and rejects all missing-fact hits | Rejects this tested embedding similarity as the final message ranker | +| Joint query-message reranker | Improves held-out recall, MRR and nDCG over both baselines, with zero no-answer false hits | Provides measured relevance ranking gains, with an expensive scoring stage | +| Question-form diagnostic | Fresh held-out MRR 1.0 and zero missing-fact false hits using manual topic anchors | Shows the value of explicit task wording; does not implement natural-language candidate retrieval | + +The joint reranker's original held-out keyword results are: + +| Search | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | +|---|---|---|---|---| +| Strict | 0.2625 | 0.3250 | 0.2192 | 0 | +| Legacy | 0.3375 | 0.4500 | 0.2993 | 80 | +| Combined + joint reranker | 0.4375 | 0.4875 | 0.4049 | 0 | + +Recall increases by 17.5 percentage points over strict and 10.0 points over legacy. The same-archive +baseline controls for adding the supplemental synthetic messages. The reranker misses the agreed +recall 0.45 and MRR 0.50 gates and the latency gate. It is an improvement on this small synthetic +benchmark, not proof of general real-chat quality or a release-ready default. + +## What retrieval, candidates, scoring and reranking mean + +1. Parse the Lucene query and resolve account authority and hard filters. +2. Retrieve possible messages with the word/stem indexes, prefixes and spelling corrections. + These messages are **candidates**: they passed retrieval eligibility, not a relevance judgment. +3. Deduplicate full message locators and merge the independently ranked lists using reciprocal + rank fusion. Its score is computed from list positions, not by a neural model. +4. In the joint-model experiment, supply the query and each candidate message together to the + model. It produces a numeric relevance score for that pair. +5. Sort the same candidates by their model scores and select the first ten. This is **reranking**; + it cannot recover a message absent from the candidate pool. + +The standard architecture is called **retrieve and rerank**, or **two-stage retrieval**. Our first +stage is lexical retrieval. A dense retriever would instead find nearby text vectors. A +**bi-encoder** independently embeds query and message and compares their vectors; a +**cross-encoder** processes the query and message jointly. The architecture and terminology are +explained in the [Sentence Transformers guide](https://sbert.net/examples/sentence_transformer/applications/retrieve_rerank/README.html). + +The tested bi-encoder is the repository's pinned multilingual e5-small, using unit vectors and +cosine similarity. The better tested cross-encoder is +[mmarco-mMiniLMv2-L12-H384-v1](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1), an +Apache-2.0 multilingual model, pinned as a roughly 119 MB quantized ONNX file plus its tokenizer +and configuration. Its configuration specifies identity activation, so our scores are raw logits, +not percentages or calibrated probabilities. Keyword ranking does not apply a score floor; +question diagnostics use a dev-selected raw score floor of 2. + +The current prototype executes one pair at a time on the CPU through the existing WebAssembly +ONNX runtime with eight threads. Its measured pair cost is about 10–12 ms: 300 fresh pairs cost +about 3–4 seconds. Loading the model costs roughly another second. These are separate stages; +the quoted scoring cost does not mean that the SQLite candidate lookup takes 3–4 seconds. +A different query requires new joint pair evaluations even when the same messages were previously +seen. An exact repeated query-message pair can reuse its score; there is no reusable standalone +message vector from this cross-encoder. Batch inference, native runtime and smaller-model options +need measurements before claiming a speed improvement. + +## What 43.8 percent means + +The labels are grade 2 for the actual answer, grade 1 for useful supporting context, and grade 0 +for distractors. Existing binary recall and MRR count grades 1 and 2 as relevant. nDCG distinguishes +them using gains 1 and 3 respectively. + +For each answerable query, recall@10 is the number of relevant messages in the first ten divided +by the number of eligible labelled relevant messages. We average these fractions equally across +queries (**macro average**). Finding one of two labelled messages gives 0.5 regardless of its +position inside the top ten. Finding both gives 1.0; finding neither gives zero. + +The unrounded held-out value is 0.4375. From the existing saved rankings, the 40 answerable queries +break down into 11 retrieving both of two relevant messages, four retrieving their sole eligible +relevant message, five retrieving one of two, and twenty retrieving none: + +```text +(11 × 1 + 4 × 1 + 5 × 0.5 + 20 × 0) / 40 = 0.4375 +``` + +Thus the fraction of queries with at least one useful result is 20/40 = **50 percent**, and the +fraction retrieving all their relevant messages is 15/40 = **37.5 percent**. These are distinct +from macro recall 43.75 percent. They are descriptive calculations from the frozen rankings, not +new inference or a tuned result. + +Ranking within the top ten is already measured: + +- **MRR@10** averages the reciprocal rank of the first relevant result: first place contributes 1, + second 1/2, tenth 1/10, and a miss zero. Here 19 queries have their first relevant hit at rank 1, + one at rank 2, and twenty miss, giving `(19 + 0.5) / 40 = 0.4875`. This is not a success rate or + an average rank. It ignores additional relevant hits after the first. +- **nDCG@10** discounts later positions, gives the actual answer more gain than supporting context, + and normalizes each query against its ideal ordering. The mean is 0.4049. It is a ranking-quality + score, not a percentage of successful queries. +- **No-answer false hits** are evaluated separately on queries with no relevant labels. Zero false + hits on eight original held-out cases does not prove general answerability; the additional missing- + fact cases use a different diagnostic and a dev-selected threshold. + +The actual-answer distinction matters. Useful support ranks first in some cases where the answer +is absent. From the existing frozen keyword rankings: useful-hit Success@1 is 47.5 percent and +Success@10 is 50.0 percent. With grade 2 alone, answer Success@1 is 30.0 percent, answer Success@3 +40.0 percent, answer Success@10 42.5 percent, and answer-only MRR@10 is 0.35625. + +## Standard evaluation terms and proposed additions + +These are **information retrieval (IR) evaluation** metrics. A test collection consists of a +corpus, queries and relevance judgments; the judgments are often called **qrels**. Saved ranked +outputs are **runs**. We use graded judgments, paired comparisons on the same queries, dev-only +parameter selection, held-out validation and ablations. References: +[Stanford IR evaluation](https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-ranked-retrieval-results-1.html), +[ir-measures definitions](https://ir-measur.es/en/latest/measures.html), and +[TREC](https://trec.nist.gov/overview.html). + +The following are proposals for the next report, not new release gates or implemented runner fields: + +| Measure | Question it answers | +|---|---| +| Success@1/3/5/10, also called hit rate | Does at least one relevant message occur within the first k? | +| Grade-2 Success and MRR | How reliably and how early do we retrieve the actual answer? | +| Recall@1/3/5/10 and candidate recall@50/100/300 | How much labelled evidence survives each stage and cutoff? | +| nDCG@3/10 | Are the strongest answers ahead of weaker support and noise? | +| MAP@10 | How early do all relevant messages appear, using binary relevance? | +| Precision@k and returned-result precision | How much of the displayed list is useful? Specify the denominator for short lists. | +| All-relevant@k | Do we retrieve every labelled relevant message? This is a task diagnostic, not our recall definition. | +| No-answer query hit rate and false-hit count | How often does an unanswerable query produce hits, and how many? | +| Per-category and per-language breakdowns | Do aggregate gains hide typo, phrase, filter or Cyrillic regressions? | +| Stage p50/p95 latency and memory | What do lookup, model load and inference each cost, cold and warm? | + +Fixed-denominator P@10 is useful only with context: most queries label just two messages, so even +a perfect top ten cannot exceed 20 percent precision when remaining slots are counted. The four +single-answer sender controls reduce this set's ideal macro P@10 to 19 percent. Returned-result +precision is different and can be 100 percent for a short list; report result count beside it. + +For an agent looking for an answer, proposed headline measures are grade-2 Success@1/3/10, graded +nDCG@10, evidence recall@10, no-answer behavior and latency. Retain supporting-evidence metrics +alongside them rather than silently changing the old definition or relabelling the corpus. + +More metrics alone do not create stronger evidence. There are only four topic families per +original split, with many correlated query variants. A larger fresh held-out set, varied numbers +of relevant messages, paraphrases, harder missing-fact negatives and independent label review are +more valuable than treating forty queries as forty independent samples. Future uncertainty estimates +should resample topic families, and report paired per-query gains/regressions and counts. + +## Next work + +Explain and settle the measurement definitions before modifying metric runners or tuning ranking. +Then derive the expanded dashboard from saved runs without repeating inference, and cross-check +standard metrics with a conventional evaluator using our explicit relevance/gain/cutoff settings. +Optimize CPU execution separately from retrieval depth: reducing depth from 300 to 50 already +loses labelled candidates before any reranker can help. Ranking changes require a fresh held-out +set; implementation-only speed changes require equivalence checks and separate timing measurements. diff --git a/docs/dev/combined-search.md b/docs/dev/combined-search.md index cd1e7ee93..c7e154b07 100644 --- a/docs/dev/combined-search.md +++ b/docs/dev/combined-search.md @@ -5,6 +5,9 @@ ranking fail the dev quality gates; joint query-message ranking improves the mea CLI/MCP and SDK defaults remain unchanged while latency and the original held-out quality gates remain unmet. Natural-language question retrieval is a separate diagnostic, not a public feature. +The [evaluation notes](combined-search-evaluation.md) record the contribution of each experiment, +the retrieval/scoring pipeline, metric interpretation and proposed next measurements. + ## Evidence and acceptance targets The [message benchmark](../../bench/message-search-quality/README.md) uses 2,616 synthetic messages, From a5d72ce81c5957f047e1b10653837a4dba8821ad Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 18:40:23 +0200 Subject: [PATCH 06/14] docs(search): research modern rankers and measure model resources --- bench/message-search-quality/README.md | 5 + bench/message-search-quality/resources.json | 1 + bench/message-search-quality/resources.ts | 48 ++++++ docs/dev/combined-search-evaluation.md | 3 + docs/dev/combined-search-model-research.md | 168 ++++++++++++++++++++ 5 files changed, 225 insertions(+) create mode 100644 bench/message-search-quality/resources.json create mode 100644 bench/message-search-quality/resources.ts create mode 100644 docs/dev/combined-search-model-research.md diff --git a/bench/message-search-quality/README.md b/bench/message-search-quality/README.md index 9072fd74f..1d4078d76 100644 --- a/bench/message-search-quality/README.md +++ b/bench/message-search-quality/README.md @@ -200,3 +200,8 @@ what each experiment contributed and proposed next metrics. In the original held report, macro recall 0.4375 means an average 43.75 percent of labelled relevant evidence retrieved; Success@10 is 50 percent, and grade-2 actual-answer Success@10 is 42.5 percent. Those last two values are descriptive audits of saved rankings, not new model runs or changed release gates. + +The [model research](../../docs/dev/combined-search-model-research.md) records alternative model +architectures and selected checkpoint sizes. [resources.ts](resources.ts) and [resources.json](resources.json) +measure the current model's isolated process RAM and scoring time; checkpoint bytes and RAM are +separate quantities. No real store is opened by that probe. diff --git a/bench/message-search-quality/resources.json b/bench/message-search-quality/resources.json new file mode 100644 index 000000000..c17de332c --- /dev/null +++ b/bench/message-search-quality/resources.json @@ -0,0 +1 @@ +{"model":{"id":"cross-encoder/mmarco-mMiniLMv2-L12-H384-v1","revision":"1427fd652930e4ba29e8149678df786c240d8825","licence":"Apache-2.0","files":[{"name":"onnx/model_quint8_avx2.onnx","sha256":"6c2513767fb63d008a4377bef7a7a3555433d9436342bb53e35a3a72ffc52d4b"},{"name":"tokenizer.json","sha256":"62c24cdc13d4c9952d63718d6c9fa4c287974249e16b7ade6d5a85e7bbb75626"},{"name":"tokenizer_config.json","sha256":"e7fbfbfa6347b4e414c1cee50d142e2c2f9a895dad68b068ae83a8b564c3837e"},{"name":"config.json","sha256":"cc2cfe51aa3fd759d21d21acf5dfd6994aa67a3c9210636d22e143699d336c77"}],"threads":8},"node":"v24.19.0","before":{"rss":70090752,"heapTotal":11898880,"heapUsed":10242648,"external":11704910,"arrayBuffers":143651},"afterLoad":{"rss":1055674368,"heapTotal":317984768,"heapUsed":178696576,"external":156148641,"arrayBuffers":147309097},"afterFirst":{"rss":1076772864,"heapTotal":318246912,"heapUsed":179462008,"external":156149173,"arrayBuffers":147309629},"after300":{"rss":1189621760,"heapTotal":318246912,"heapUsed":199818576,"external":127710196,"arrayBuffers":118870652},"peakObservedRss":1317421056,"processHighWaterBytes":1331576832,"modelIncrementAtLoad":985583616,"modelIncrementAfter300":1119531008,"loadMs":1270.7682869999999,"scoring300Ms":3247.012192,"score":8.382225036621094,"notes":"Entire isolated Node process RSS includes runtime, tokenizer, model weights and buffers; no store or account opened."} diff --git a/bench/message-search-quality/resources.ts b/bench/message-search-quality/resources.ts new file mode 100644 index 000000000..ed1ad6c11 --- /dev/null +++ b/bench/message-search-quality/resources.ts @@ -0,0 +1,48 @@ +import { readFileSync, existsSync } from "node:fs" +import { performance } from "node:perf_hooks" +import { openCrossEncoder } from "./crossencoder.ts" +const before = process.memoryUsage() +const started = performance.now() +const directory = process.env.MESSAGE_RERANKER_DIR +if (!directory) throw new Error("MESSAGE_RERANKER_DIR must name the separately prepared local model") +const ranker = await openCrossEncoder(directory) +const afterLoad = process.memoryUsage() +const loadMs = performance.now() - started +const modelScore = await ranker.score( + "What rollback decision was agreed for the Aurora rollout?", + "Aurora rollout rollback decision: disable the canary and restore build 17.", +) +const afterFirst = process.memoryUsage() +let peakRss = afterFirst.rss +const start = performance.now() +for (let i = 0; i < 300; i++) { + await ranker.score( + "What rollback decision was agreed for the Aurora rollout?", + `Aurora rollout rollback: agenda only; no decision yet. Thread ${i}.`, + ) + peakRss = Math.max(peakRss, process.memoryUsage().rss) +} +const after300 = process.memoryUsage() +const scoring300Ms = performance.now() - start +const highWaterKiB = existsSync("/proc/self/status") + ? Number(/^VmHWM:\s+(\d+)/m.exec(readFileSync("/proc/self/status", "utf8"))?.[1]) + : process.resourceUsage().maxRSS +await ranker.close() +const report = { + model: ranker.model, + node: process.version, + before, + afterLoad, + afterFirst, + after300, + peakObservedRss: peakRss, + processHighWaterBytes: highWaterKiB * 1024, + modelIncrementAtLoad: afterLoad.rss - before.rss, + modelIncrementAfter300: after300.rss - before.rss, + loadMs, + scoring300Ms, + score: modelScore, + notes: + "Entire isolated Node process RSS includes runtime, tokenizer, model weights and buffers; no store or account opened.", +} +console.log(JSON.stringify(report)) diff --git a/docs/dev/combined-search-evaluation.md b/docs/dev/combined-search-evaluation.md index c21b9e987..28bfbd46b 100644 --- a/docs/dev/combined-search-evaluation.md +++ b/docs/dev/combined-search-evaluation.md @@ -147,6 +147,9 @@ of relevant messages, paraphrases, harder missing-fact negatives and independent more valuable than treating forty queries as forty independent samples. Future uncertainty estimates should resample topic families, and report paired per-query gains/regressions and counts. +The [model and architecture research](combined-search-model-research.md) compares modern rerankers, +open System One alternatives, candidate cascades and the measured disk/RAM/download costs. + ## Next work Explain and settle the measurement definitions before modifying metric runners or tuning ranking. diff --git a/docs/dev/combined-search-model-research.md b/docs/dev/combined-search-model-research.md new file mode 100644 index 000000000..e3142a7fc --- /dev/null +++ b/docs/dev/combined-search-model-research.md @@ -0,0 +1,168 @@ +# Local ranking models and decision layers + +This research supports the [combined-search design](combined-search.md) and +[evaluation notes](combined-search-evaluation.md). It proposes experiments rather than a public +model/default change. The current model is measured locally; alternative model sizes come from +publisher model cards and public Hub file metadata. Published GPU timings are not laptop CPU timings. + +## Ways to use the candidate pool + +The current dev pool contains 90 percent of labelled relevant messages at depth 300; every +non-paraphrase category has full candidate recall and paraphrases have zero. It does not mean that +90 percent of the 300 returned messages are relevant. Most are distractors. Candidate recall is a +coverage measure, not precision, and these pool measurements are on dev, not a claimed universal rate. + +A proposed cascade keeps wide retrieval but spends expensive inference selectively: + +```mermaid +flowchart LR + A[Lucene filters and forgiving retrieval] --> B[Wide candidate pool] + B --> C[Cheap ranking and duplicate grouping] + C --> D[Strong reranking of a smaller shortlist] + D --> E[Answer / support / no-answer decision] + E --> F[Labelled message results] +``` + +The smaller shortlist must be chosen by an improved cheap stage, not merely by truncating the +current order: dev candidate recall drops from 0.90 at 300 to 0.4375 at 50. Measure actual-answer +and evidence recall before/after every stage. Grouping should suppress near-identical boilerplate +without discarding genuinely distinct evidence; simple chat caps alone did not improve this corpus. + +Ranking asks which message is relatively better. An answerability decision asks whether the +requested fact is actually present. Keep those separate. A decision model could classify five to +ten finalists as direct answer, support, missing fact, contradictory/outdated, or needing thread +context. Include an explicit no-answer option; choosing among supplied candidates alone can force +an answer even when every candidate is poor. Thresholds need our own held-out calibration. + +Useful candidates should retain matcher provenance, account/chat authority, corrected spellings +and model/version metadata. Returning only three hits does not itself remove the cost of scoring +300. Adaptive expansion or early stopping requires measured miss rates; a high score on an already +examined hit does not establish that unexamined messages contain no better answer. + +## What Jev contributes + +[TypeSafe's introduction](https://typesafe.ai/blog/introducing-system-one-models-and-jev) describes +shared-state typed decisions and non-autoregressive outputs, with vendor-reported 70–500 ms calls. +[The model documentation](https://docs.typesafe.ai/models) describes a hosted API with shared state +and parallel questions, a 64k combined input budget and a 32k state-plus-longest-question budget. +The introduction specifies up to 255 choice options. No public local weights or model size were +found in those official materials. These timings are not measurements on our 300-message workload. + +Our current cross-encoder already emits a scalar without generating text. The interesting System +One features for this task are shared context, typed evidence decisions and uncertainty handling. +Valid output types do not establish factual correctness. A single forward pass still has to process +the supplied text; long lists can have substantial computation and memory costs. + +Open alternatives with publisher-owned repositories: + +- [Laya](https://github.com/NandhaKishorM/laya): Apache-2.0 typed decision models. Its multilingual + checkpoint uses mmBERT-base, about 322M parameters, with an ONNX option and a TypeScript project. + The multilingual default input limit is 1,024 tokens; the documented extension is 8,192. The + advertised 33 ms figure is a GPU result, not our CPU latency. +- [Kev](https://github.com/jaredpalmer/kev): Apache-2.0 shared-state models based on Qwen, starting + at 0.8B. [Kev-0.8B](https://huggingface.co/jaredpalmer/kev-0.8b) is English-only according to its + model card. Its roughly 43 MB adapter also needs the approximately 1.75 GB BF16 base and a head. + CUDA/MLX benchmarks do not establish performance on our Linux CPU. +- [CLM](https://github.com/Contrastive-LM/CLM): separately caches state/action representations and + exposes direct candidate ranking. The reference uses a Qwen3-8B encoder plus projection heads; + its small head download is not the complete model. This is a useful architecture reference, + but the published reference is not a small CPU-first replacement. + +For a Russian/English local decision experiment, Laya-multilingual is the more appropriate first +candidate than Kev-0.8B. This is a suitability judgment from declared language, footprint and +interface, not a benchmark result proving superior search quality. + +## Ranking model shortlist + +Sizes below are selected weight files, decimal MB/GB, excluding tokenizer/config/runtime unless +explicitly stated. Only the current model's RAM has been measured. Quantization may affect ranking +and must be evaluated; different models' scores or thresholds are not interchangeable. + +| Model | Published footprint | Proposed role and constraints | +|---|---|---| +| Current multilingual mMARCO MiniLMv2 | 118.6 MB int8 ONNX; 135.7 MB with tokenizer/config | Controlled baseline; first test native and batched execution | +| [TinyBERT-L2](https://huggingface.co/cross-encoder/ms-marco-TinyBERT-L2-v2) | 4.4M parameters; 4.52 MB int8 ONNX | Very small English speed control; older model, not a multilingual replacement | +| [MiniLM-L6](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2) | 22.7M; 23.2 MB int8 ONNX | English CPU baseline between TinyBERT and the current multilingual model | +| [GTE multilingual reranker](https://huggingface.co/Alibaba-NLP/gte-multilingual-reranker-base) | About 306M; 612 MB FP16 | Apache-2.0 encoder-only reranker covering Russian and 70+ languages; CPU serving is documented | +| [Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) | 596M; 1.19 GB BF16 | Apache-2.0 multilingual, instruction-conditioned quality comparator; bigger does not establish faster CPU inference | +| [Jina reranker v3.5](https://huggingface.co/jinaai/jina-reranker-v3.5) | 597M; 1.19 GB BF16; official Q4 GGUF about 397 MB plus projector | Modern multilingual listwise ranking; published speed improvements use A100/FlashAttention; CC BY-NC 4.0 limits default commercial adoption | +| [Laya-multilingual](https://huggingface.co/convaiinnovations/laya-multilingual) | 322M; 644 MB safetensors | Apache-2.0 typed finalist/answerability experiment, not a tested drop-in reranker | +| [Answer.AI ColBERT small](https://huggingface.co/answerdotai/answerai-colbert-small-v1) | 33.4M; about 34 MB int8 ONNX | English late-interaction architecture control with reusable message representations | + +[Mixedbread xsmall](https://huggingface.co/mixedbread-ai/mxbai-rerank-xsmall-v1) is another compact +English comparator: 70.8M parameters, about 87 MB quantized ONNX. The +[FlashRank project](https://github.com/PrithivirajDamodaran/FlashRank) demonstrates CPU use of small +ONNX rerankers; its tiny model is the TinyBERT family, not evidence that every multilingual model +can fit in 4 MB. + +[GLiNER2.5](https://github.com/fastino-ai/GLiNER2) is an additional typed classification/extraction +route: 74M small English and 287M multilingual checkpoints. It could supply evidence-role features, +but its declared task is extraction/classification; message relevance and answerability need validation. + +## Listwise and late-interaction alternatives + +Jina v3.5 jointly ranks lists and shares query/context computation; it is architecturally different +from 300 independent pair passes. Its model card reports 1.22–1.56x improvements over v3 on A100, +not a measured speedup over our model or on our CPU. Evaluate document-order sensitivity, list +length, batch merging and RAM; raw scores can depend on the surrounding candidate list. Its +noncommercial weight license makes it a research comparator rather than the default permissive option. + +ColBERT-style **late interaction** precomputes token-level message representations. A new query is +encoded once; MaxSim matches its token vectors against stored message vectors. This shifts neural +message encoding into archive preparation and can avoid online neural inference for every pair. +It adds index storage, freshness/permission handling and build costs. It is different from the +single-vector e5 cosine experiment, so e5's failure does not rule it out. + +[ModernColBERT](https://huggingface.co/lightonai/GTE-ModernColBERT-v1) is an Apache-2.0 English example, +about 149M parameters and a 150 MB int8 export. +[Liquid LFM2.5-ColBERT-350M](https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M) is a newer efficient +multilingual example with about 353M parameters, but its eleven declared languages omit Russian, +and its license is the LFM Open License. Neither should silently become the Russian default. +A suitable multilingual checkpoint and its index costs need separate evaluation. + +## Current measured disk, RAM and preparation + +[resources.ts](../../bench/message-search-quality/resources.ts) loads the existing pinned model in +an isolated Node process and scores 300 short synthetic messages; no store/account is opened. +[resources.json](../../bench/message-search-quality/resources.json) records the model pins, whole- +process RSS at each stage, high-water resident memory, model load and inference times. + +The measured process starts at about 70 MB RSS, uses about 1.06 GB after load and peaks at about +1.33 GB. Scoring 300 short pairs takes about 3.25 seconds; load takes about 1.27 seconds. These are +lab observations for this runtime and workload, not a model-only allocation breakdown or an upper +bound for larger contexts/batches. The resident footprint includes tokenizer objects, ONNX engine, +weights, worker state and intermediate buffers; disk weight size alone does not predict RAM. + +Reproduce after explicit model preparation: + +```sh +MESSAGE_RERANKER_DIR=/tmp/message-search-reranker-1427fd6 \ + node bench/message-search-quality/resources.ts > bench/message-search-quality/resources.json +``` + +The existing [preparation script](../../bench/message-search-quality/prepare-reranker.py) downloads +only the four pinned files in [the manifest](../../bench/message-search-quality/reranker.json), +verifies SHA-256, writes partial downloads separately, and reuses verified files. It does not clone +all precision variants from the model repository. The offline benchmark never installs a model. +A production model category and explicit installer remain future work. + +Download time depends on transfer rate. For the complete 135.7 MB set, transfer-only estimates +are about 109 seconds at 10 Mbit/s, 54 seconds at 20 Mbit/s and 11 seconds at 100 Mbit/s. Add network +setup, server delays and verification. The original download was not timed. A 1.2 GB checkpoint +would take about eight minutes at 20 Mbit/s before that overhead. RAM for alternative models is +unmeasured; checkpoint precision, runtime, input length, batching and accelerator placement matter. + +## Recommended experiment order + +1. Compare the current weights under native CPU execution and length-grouped batches, with ranking + equivalence, memory and cold/warm latency checks. This isolates implementation cost from model quality. +2. Add a tiny English control and one permissively licensed multilingual reranker, then evaluate a + fast-to-strong cascade. Keep the 300-message retrieval pool until shortlist recall proves equivalent. +3. Test Laya-multilingual on a few finalists for actual answer/support/none and context-needed decisions. + Measure answer-only success, false hits, calibration and the extra end-to-end latency. +4. Investigate a multilingual late-interaction index if recurrent online pair inference remains too costly. + Jina listwise ranking is a useful separately licensed architecture comparator. + +No new alternative weights were installed, no hosted Jev call was made, and no public search behavior +changed during this research. Future selection uses fresh held-out families rather than retuning +against the already exposed keyword/question results. From 8ccdea2143132a6144de327aa0b57dd2ace0421e Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 18:44:42 +0200 Subject: [PATCH 07/14] docs(search): include semantic candidate generation in research plan --- docs/dev/combined-search-model-research.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/dev/combined-search-model-research.md b/docs/dev/combined-search-model-research.md index e3142a7fc..ccb6c122a 100644 --- a/docs/dev/combined-search-model-research.md +++ b/docs/dev/combined-search-model-research.md @@ -25,7 +25,11 @@ flowchart LR The smaller shortlist must be chosen by an improved cheap stage, not merely by truncating the current order: dev candidate recall drops from 0.90 at 300 to 0.4375 at 50. Measure actual-answer -and evidence recall before/after every stage. Grouping should suppress near-identical boilerplate +and evidence recall before/after every stage. A separate candidate-generation experiment can +union lexical hits with semantic hits, including existing conversation chunks mapped back to +messages, while preserving authority and hard filters. This could address the paraphrase misses. +The e5 cosine result rejects it as our final reranker; it does not measure or rule out that +candidate-generation role. Noise and no-answer behavior must be evaluated for the full cascade. Grouping should suppress near-identical boilerplate without discarding genuinely distinct evidence; simple chat caps alone did not improve this corpus. Ranking asks which message is relatively better. An answerability decision asks whether the From 3a4e5303c4b6c630d4f682d4a90f44294eb73ce3 Mon Sep 17 00:00:00 2001 From: leemour Date: Fri, 9 Oct 2026 19:56:36 +0200 Subject: [PATCH 08/14] test(search): transfer ranking research to cli-testing --- CLEANUP.md | 2 + bench/message-search-quality/README.md | 207 - .../augmented-baseline.json | 11550 ------ bench/message-search-quality/baseline.json | 11549 ------ .../message-search-quality/combined-dev.json | 3723 -- bench/message-search-quality/combined.ts | 184 - bench/message-search-quality/corpus.json | 19452 ---------- .../crossencoder-dev.json | 30187 ---------------- bench/message-search-quality/crossencoder.ts | 66 - .../generate-questions.py | 50 - bench/message-search-quality/generate.py | 60 - .../prepare-reranker.py | 25 - bench/message-search-quality/questions.json | 4544 --- bench/message-search-quality/reranker.json | 28 - bench/message-search-quality/resources.json | 1 - bench/message-search-quality/resources.ts | 48 - bench/message-search-quality/run.ts | 137 - .../message-search-quality/semantic-dev.json | 27423 -------------- bench/message-search-quality/semantic.ts | 422 - bench/message-search-quality/tsconfig.json | 13 - docs/dev/combined-search-evaluation.md | 160 - docs/dev/combined-search-handoff.md | 4 +- docs/dev/combined-search-model-research.md | 172 - docs/dev/combined-search.md | 12 +- 24 files changed, 10 insertions(+), 110009 deletions(-) delete mode 100644 bench/message-search-quality/README.md delete mode 100644 bench/message-search-quality/augmented-baseline.json delete mode 100644 bench/message-search-quality/baseline.json delete mode 100644 bench/message-search-quality/combined-dev.json delete mode 100644 bench/message-search-quality/combined.ts delete mode 100644 bench/message-search-quality/corpus.json delete mode 100644 bench/message-search-quality/crossencoder-dev.json delete mode 100644 bench/message-search-quality/crossencoder.ts delete mode 100644 bench/message-search-quality/generate-questions.py delete mode 100644 bench/message-search-quality/generate.py delete mode 100644 bench/message-search-quality/prepare-reranker.py delete mode 100644 bench/message-search-quality/questions.json delete mode 100644 bench/message-search-quality/reranker.json delete mode 100644 bench/message-search-quality/resources.json delete mode 100644 bench/message-search-quality/resources.ts delete mode 100644 bench/message-search-quality/run.ts delete mode 100644 bench/message-search-quality/semantic-dev.json delete mode 100644 bench/message-search-quality/semantic.ts delete mode 100644 bench/message-search-quality/tsconfig.json delete mode 100644 docs/dev/combined-search-evaluation.md delete mode 100644 docs/dev/combined-search-model-research.md diff --git a/CLEANUP.md b/CLEANUP.md index 458b8d73b..c69fa6ebd 100644 --- a/CLEANUP.md +++ b/CLEANUP.md @@ -1,3 +1,5 @@ # Cleanup - `scripts/parity-convert.ts` — the one-off rewrite of `parity.json` into the N-CLI shape; delete once no open branch edits the old shape (feat/parity-n-clis). 2026-10-01 + +- `bench/message-search-quality/`, `docs/dev/combined-search-evaluation.md`, `docs/dev/combined-search-model-research.md` — owner requested transfer to cli-testing/performance/search; copied and hash-verified before removal in this batch. 2026-10-09 diff --git a/bench/message-search-quality/README.md b/bench/message-search-quality/README.md deleted file mode 100644 index 1d4078d76..000000000 --- a/bench/message-search-quality/README.md +++ /dev/null @@ -1,207 +0,0 @@ -# Message search ranking quality - -This offline benchmark compares the production strict Lucene search (including its default -stemming) and legacy fallback chain through `searchStore`. It contains 2,616 invented messages -in nine chats, and 96 labelled queries: 40 answerable and eight no-answer per split. Each topic -has two useful messages and 300 newer distractors. The busy operations chat holds 1,600 topic -distractors plus 200 unrelated messages; eight topic chats each hold 102 messages. - -The committed [corpus.json](corpus.json) is the evaluation input. [generate.py](generate.py) -reproduces it deterministically with no packages or network. No real account, messenger adapter, -model cache or network request is used. The runner opens an explicit new temporary SQLite file, -points `MESSAGING_STORE` and the model cache into that temporary directory, builds both indexes, -and closes the store in `finally`. It keeps the temporary files for inspection; removal is a -separate owner-approved cleanup task. - -## Reproduce - -From the repository root, with dependencies installed: - -```sh -pnpm build -pnpm exec tsc -p bench/message-search-quality/tsconfig.json -node bench/message-search-quality/run.ts > bench/message-search-quality/baseline.json -``` - -The script fixes UTC for both dialects. It exits unsuccessfully for invalid labels, duplicate -message/query ids, incomplete indexes, unknown result ids, duplicate hits or a missing measured -query/mode. Every query produces numeric metrics in both modes; answerable metrics are null -only for no-answer queries. The aggregate reports every split, mode and category. Corpus, runner -and relevant production-file hashes identify what was measured; the temporary path changes per run. -The benchmark runs outside `pnpm test`; its own TypeScript check is required because the repository -lint and TypeScript configuration exclude this folder. - -## Labels and limits - -Queries cover whole words, typos, word beginnings, Cyrillic and Latin, phrases, sender/chat/date -filters, AND/OR, paraphrases, and absent/near-miss no-answer requests. Each query states its intent. -Grade 2 is the message answering the decision/policy question; grade 1 is its useful owner or -confirmation message. Templates, unanswered questions and rejected proposals are grade 0. -The sender-filter queries label only the grade-2 message because the confirmation has another sender. -All other answerable queries label both useful messages. No-answer requests have no labels. - -Lucene `date:2026-10-02` is paired with legacy `after:2026-10-02 before:2026-10-03`. -Explicit Lucene AND is paired with legacy implicit conjunction. These are handwritten equivalents; -the runner does not reinterpret an unsupported dialect silently. OR is limited to the same -precedence-compatible shape in both parsers. The `exact` flag is not enabled: the Lucene baseline -measures the existing default strict service, with stems ready. - -Dev topics are Aurora, Cedar, Север and Lumen; held-out topics are Harbor, Maple, Вектор and Prism. -The topic families and labelled messages do not overlap, but both are indexed in the same archive. -Templates and query categories are shared. This is a small synthetic holdout, not a blinded external -study or evidence of general multilingual quality. Held-out results are reported for baseline -transparency; future parameter choices must use dev only. A lexical reranker cannot reliably -recognize an agreed decision merely from shared topic words. Paraphrases remain explicit misses -in a lexical-only release; do not redefine their labels to inflate its score. - -Recall@10 is the fraction of labelled messages in the first ten hits, macro-averaged over -answerable queries. MRR uses the first grade-1-or-2 hit within the returned ten (zero for a miss). -nDCG@10 uses gain `2^grade - 1` and a discount of `log2(rank + 1)`, normalized against the sorted -labels. No-answer false hits count all returned messages on no-answer queries, up to ten each; -query hit rate is the fraction of no-answer queries returning any hit. These are retrieval false -hits, not claims about an agent's eventual answer. Raw ids, labels, match types, scores and -corrections are retained in [baseline.json](baseline.json), without real messages. - -## Baseline - -| Split | Mode | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | No-answer query hit rate | -|---|---|---|---|---|---|---| -| dev | Lucene | 0.313 | 0.325 | 0.287 | 0 / 80 | 0.000 | -| dev | legacy | 0.388 | 0.425 | 0.354 | 80 / 80 | 1.000 | -| held-out | Lucene | 0.263 | 0.325 | 0.219 | 0 / 80 | 0.000 | -| held-out | legacy | 0.338 | 0.450 | 0.299 | 80 / 80 | 1.000 | - -The strict matcher misses typos and word beginnings. Legacy recovers some, but its early successful -list prevents other matchers from contributing; broad any-word fallback fills no-answer pages. -Both can bury useful messages beneath exact keyword matches. Sender/date filters are easy controls -and achieve full recall; category reports prevent them from concealing poor unfiltered rankings. - -## Combined lexical experiment - -After building, run the separate dev evaluation: - -```sh -pnpm exec tsc -p bench/message-search-quality/tsconfig.json -node bench/message-search-quality/combined.ts > bench/message-search-quality/combined-dev.json -``` - -It calls the internal production matcher directly without changing public `searchStore` defaults. -All 180 combinations of candidate depth, RRF constant, lexical reranking signal and chat cap are -measured on dev only. All returned rankings are scored against the unchanged corpus labels. -Explicit AND/OR/phrase query ids must match the strict production top ten exactly. The runner -never supplies query `intent` or relevance labels to the matcher. - -Every tested configuration has dev recall@10 0.3625, MRR@10 0.375 and nDCG@10 0.3264, with zero -no-answer false hits. None meets the approved 0.45/0.50/0.40 gates. [combined-dev.json](combined-dev.json) -records each configuration, measured timing and hashes, and the diagnostic dev rankings. No -combined held-out result is produced when dev fails. The original baseline is preserved. -The [revised proposal](../../docs/dev/combined-search.md#measured-lexical-experiment-and-revised-proposal) -requests a semantic evaluation before public adoption. Timings are from this small corpus on a -shared machine; they are not release-scale latency evidence. - -## Candidate recall and semantic ranking - -[semantic.ts](semantic.ts) measures the lexical candidate pool before reranking. The original -corpus and labels remain unchanged. [questions.json](questions.json) adds eight answerable and four -no-answer questions on the existing dev topics, plus four fresh held-out topic families with 608 -new synthetic messages and twelve questions. [generate-questions.py](generate-questions.py) -reproduces this supplement. Its input hashes are frozen before model inference and parameter selection. -Query `intent` and relevance labels are never supplied to either model. - -Dev candidate recall is 0.4375 at depth 50, 0.7500 at 100, and 0.9000 at 300/500. Depth 300 is the -smallest tested pool reaching maximum recall. Every non-paraphrase category has full candidate -recall at that depth; paraphrases have zero. The pool size measures returned candidates from the -internal service, rather than the union of every raw matcher list. - -To measure the already installed, pinned e5-small model: - -```sh -pnpm build -pnpm exec tsc -p bench/message-search-quality/tsconfig.json -node bench/message-search-quality/semantic.ts > bench/message-search-quality/semantic-dev.json -``` - -The benchmark verifies the local model's hashes before opening it. e5-small cosine reranking -reduces original dev keyword recall@10 from 0.3625 to 0.2750, MRR from 0.3750 to 0.3000, and nDCG -from 0.3264 to 0.2674. On the additional answerable questions it improves ranking when supplied -manual topic anchors for retrieval, but every tested threshold preserving zero no-answer hits -reduces answerable recall to zero. No e5 held-out evaluation is performed because dev fails. - -The second experiment uses the official -[mmarco-mMiniLMv2-L12-H384-v1 model](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1) -to jointly score query-message pairs. [reranker.json](reranker.json) pins an Apache-2.0 ONNX export, -its revision, tokenizer/config files and SHA-256 hashes. Its identity activation is retained: -scores are raw logits, not cosine or calibrated probabilities. The benchmark-only adapter is -[crossencoder.ts](crossencoder.ts); it uses the installed tokenizer and WebAssembly ONNX runtime, -checks input/output shapes and rejects overlength pairs instead of silently truncating them. - -Model preparation is an explicit, separate network step, totaling about 136 MB. Use a directory -outside the real store and repository, then run the offline evaluation: - -```sh -python bench/message-search-quality/prepare-reranker.py /tmp/message-search-reranker-1427fd6 -MESSAGE_RERANKER_DIR=/tmp/message-search-reranker-1427fd6 \ - node bench/message-search-quality/semantic.ts --crossencoder > bench/message-search-quality/crossencoder-dev.json -``` - -The evaluation cannot download or install a model. Preparation and benchmark stores retain their -files for inspection; cleanup is a separate task. Both models score only bounded lexical candidates. -Explicit Boolean/phrase/AST requests retain the existing strict ranking. All parameters are selected -on dev; held-out inference happens only after the dev ranking/no-answer gates pass. - -The question-form experiment deliberately separates two measurements: direct full-question Lucene -retrieval, which currently finds no candidates, and reranking a manually supplied topic shortlist. -Manual topic anchors are diagnostic inputs, not an implemented natural-language query parser. -These results do not establish end-to-end question search. Questions asking for an unknown auditor -provide harder no-answer controls than the original requests with absent topic words. - -### Measured query-message ranking - -[crossencoder-dev.json](crossencoder-dev.json) records the frozen model, depth 300, raw score floor -2 selected for the question diagnostic, exact hashes, per-query rankings, and both held-out reports. -Keyword requests use reranking without a score floor; their no-answer candidate pools are empty. -The question floor is fitted on dev only and is not a general answerability or confidence guarantee. - -| Original query split | Mode | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | -|---|---|---|---|---|---| -| dev | Strict | 0.313 | 0.325 | 0.287 | 0 | -| dev | Legacy | 0.388 | 0.425 | 0.354 | 80 | -| dev | Combined + joint reranker | 0.525 | 0.553 | 0.477 | 0 | -| held-out | Strict | 0.263 | 0.325 | 0.219 | 0 | -| held-out | Legacy | 0.338 | 0.450 | 0.299 | 80 | -| held-out | Combined + joint reranker | 0.438 | 0.488 | 0.405 | 0 | - -The original keyword held-out result beats both previous modes but misses the approved recall -0.45 and MRR 0.50 gates. nDCG and no-answer gates pass. No parameters are changed after reading -these results. The fresh question held-out diagnostic has recall 0.6875, MRR 1.0, nDCG 0.8660 and -zero false hits on four no-answer queries, using manual anchors. This small synthetic set does not -establish general answerability or a functioning full-question retrieval path. - -The semantic experiment indexes both the original corpus and fresh supplemental messages. To -control for the changed archive, strict/legacy results on that same 3,224-message archive are in -[augmented-baseline.json](augmented-baseline.json); their original-query metrics equal the old -baseline. Reproduce that comparison without changing the original baseline file: - -```sh -node bench/message-search-quality/run.ts --include-questions > bench/message-search-quality/augmented-baseline.json -``` - -The joint model scores about 10–12 ms per pair in these runs on eight WebAssembly threads, plus -roughly a second to load. Scoring 300 uncached pairs therefore costs about 3–4 seconds, exceeding -the proposed 250 ms latency gate. This is measured benchmark inference, not production p95 or -an optimized batch/native implementation. Exact-pair caching helps repeated identical requests; -it does not remove the cost of a new query. No public service/default or consumer version changed. - -## Interpreting the measurements - -[The evaluation notes](../../docs/dev/combined-search-evaluation.md) explain candidate retrieval, -model scoring, reranking cost and the precise recall/MRR/nDCG denominators. They also record -what each experiment contributed and proposed next metrics. In the original held-out keyword -report, macro recall 0.4375 means an average 43.75 percent of labelled relevant evidence retrieved; -Success@10 is 50 percent, and grade-2 actual-answer Success@10 is 42.5 percent. Those last two -values are descriptive audits of saved rankings, not new model runs or changed release gates. - -The [model research](../../docs/dev/combined-search-model-research.md) records alternative model -architectures and selected checkpoint sizes. [resources.ts](resources.ts) and [resources.json](resources.json) -measure the current model's isolated process RAM and scoring time; checkpoint bytes and RAM are -separate quantities. No real store is opened by that probe. diff --git a/bench/message-search-quality/augmented-baseline.json b/bench/message-search-quality/augmented-baseline.json deleted file mode 100644 index cae2e41f6..000000000 --- a/bench/message-search-quality/augmented-baseline.json +++ /dev/null @@ -1,11550 +0,0 @@ -{ - "schemaVersion": 1, - "k": 10, - "node": "v24.19.0", - "timezone": "UTC", - "storePath": "/tmp/message-search-quality-MkWCIS/quality.db", - "archiveSupplementSha256": "390ff1475964d982a06afaf9e2f6f840f5b8304fcffe58ebedd5a0ea8d54478e", - "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", - "runnerSha256": "4b6fe3eccec01b1f95704b4447aa7647070cbc213fd524f6f665b630ca7ef507", - "productionSha256": { - "services/messages.js": "62f927cddd912275bb46a7c990d64881c2b4a22eac12d54f484fd5ca4223a3e4", - "services/messages-search.js": "35eb4ed1dcc002a80985327ffa9f6bd25a5ee6b1a31f3024203a38b6605c7d2a", - "search/search.js": "e1c7975b1573936515244fdc60454a699552afde89363b517d35974e115384b6", - "search/query.js": "89a7030d749f06ca61b9867bf29db4244617943e14c383aa5a09fd40ed819772", - "search/correct.js": "54e5089f217840dd9e27e6e89c29486c2d8704850af16dafd1b8410998066270", - "search/stem.js": "84475baa966a17595c480015668dc77da60409a00c391aa9ca6a718433386007", - "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", - "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964", - "store/sqlite/words.js": "be7c86aa22f8a308ce2fba664801ffd65da4aad444f1a8e62533548980a9e5b3" - }, - "messages": 3224, - "queries": 96, - "metrics": { - "dev": { - "lucene": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3125, - "mrr": 0.325, - "ndcgAt10": 0.2865560469084554, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.25, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.9491768952476267, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - } - } - }, - "legacy": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3875, - "mrr": 0.425, - "ndcgAt10": 0.3539325811955994, - "noAnswerFalseHits": 80, - "noAnswerQueryHitRate": 1, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.25, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.9491768952476267, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.5, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - } - } - } - }, - "test": { - "lucene": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.2625, - "mrr": 0.325, - "ndcgAt10": 0.21917951262131133, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.8475306857428799, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - } - } - }, - "legacy": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3375, - "mrr": 0.45, - "ndcgAt10": 0.2992618230965487, - "noAnswerFalseHits": 80, - "noAnswerQueryHitRate": 1, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.8475306857428799, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.5, - "mrr": 0.75, - "ndcgAt10": 0.66311732856428, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - } - } - } - } - }, - "rankings": [ - { - "id": "aurora-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "aurora rolluot rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "aurora rolluot rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10168", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10163", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10158", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10153", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10148", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10143", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10138", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10133", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10128", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - } - ], - "corrections": [ - { - "from": "rolluot", - "to": [ - "rollout" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "aurora roll rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "aurora roll rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10168", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10163", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10158", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10153", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10148", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10143", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10138", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10133", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10128", - "chatId": "999", - "match": "beginnings", - "score": 8.32129250405688 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"aurora rollout rollback\"", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"aurora rollout rollback\"", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "aurora rollout from:700", - "relevant": { - "10001": 2 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": null, - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "aurora rollout from:700", - "relevant": { - "10001": 2 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": "words", - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "aurora rollout chat:101", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "aurora rollout chat:101", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10293", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10288", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10283", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10278", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10273", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10268", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10263", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10258", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10253", - "chatId": "101", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "aurora rollout date:2026-10-02", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": null, - "score": 4.8518902533575785 - }, - { - "id": "10002", - "chatId": "101", - "match": null, - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "aurora rollout after:2026-10-02 before:2026-10-03", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": "words", - "score": 4.8518902533575785 - }, - { - "id": "10002", - "chatId": "101", - "match": "words", - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "aurora AND rollout AND rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "aurora rollout OR rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "aurora rollout OR rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "aurora deployment reversal", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "aurora deployment reversal", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "anyWord", - "score": 8.206683800405353 - }, - { - "id": "10168", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10163", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10158", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10153", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10148", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10143", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10138", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10133", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10128", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "aurora quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "aurora quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10168", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10163", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10158", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10153", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10148", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10143", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10138", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10133", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10128", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "aurora-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "aurora rollout rollback purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "aurora rollout rollback purple", - "relevant": {}, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10168", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10163", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10158", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10153", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10148", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10143", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10138", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10133", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10128", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "cedar-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "cedar bakcup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "cedar bakcup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10465", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10460", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10455", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10450", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10445", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10440", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10435", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10430", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10425", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - } - ], - "corrections": [ - { - "from": "bakcup", - "to": [ - "backup" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "cedar back retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "cedar back retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10465", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10460", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10455", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10450", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10445", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10440", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10435", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10430", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10425", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"cedar backup retention\"", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"cedar backup retention\"", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "cedar backup from:700", - "relevant": { - "10303": 2 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "cedar backup from:700", - "relevant": { - "10303": 2 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "cedar backup chat:102", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "cedar backup chat:102", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10595", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10590", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10585", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10580", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10575", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10570", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10565", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10560", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10555", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "cedar backup date:2026-10-02", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10304", - "chatId": "102", - "match": null, - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "cedar backup after:2026-10-02 before:2026-10-03", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10304", - "chatId": "102", - "match": "words", - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "cedar AND backup AND retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "cedar backup OR retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "cedar backup OR retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "cedar snapshot lifetime", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "cedar snapshot lifetime", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": "anyWord", - "score": 10.379727637637021 - }, - { - "id": "11511", - "chatId": "106", - "match": "anyWord", - "score": 7.449128259278972 - }, - { - "id": "10465", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10460", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10455", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10450", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10445", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10440", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10435", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10430", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "cedar quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "cedar quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10465", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10460", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10455", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10450", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10445", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10440", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10435", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10430", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10425", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "cedar-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "cedar backup retention purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "cedar backup retention purple", - "relevant": {}, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10465", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10460", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10455", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10450", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10445", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10440", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10435", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10430", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10425", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "север-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 8.32129250405688 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 8.32129250405688 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "север сертифкат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "север-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "север сертифкат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "corrected", - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": "corrected", - "score": 8.32129250405688 - }, - { - "id": "10767", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10762", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10757", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10752", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10747", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10742", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10737", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "10732", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - } - ], - "corrections": [ - { - "from": "сертифкат", - "to": [ - "сертификат" - ] - } - ], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "север серти обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "север-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "север серти обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "beginnings", - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": "beginnings", - "score": 8.32129250405688 - }, - { - "id": "10767", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10762", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10757", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10752", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10747", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10742", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10737", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "10732", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"север сертификат обновление\"", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 3.126652600631594 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "10862", - "chatId": "103", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "север-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"север сертификат обновление\"", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 3.126652600631594 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "10727", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "север-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "север сертификат from:700", - "relevant": { - "10605": 2 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 5.824679559744159 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "север сертификат from:700", - "relevant": { - "10605": 2 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 5.824679559744159 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "север сертификат chat:103", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 5.824679559744159 - }, - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 5.651770526756559 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "север сертификат chat:103", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 5.824679559744159 - }, - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 5.651770526756559 - }, - { - "id": "10902", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10897", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10892", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10887", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10882", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10877", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10872", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "10867", - "chatId": "103", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "север сертификат date:2026-10-02", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 5.824679559744159 - }, - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 5.651770526756559 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "север сертификат after:2026-10-02 before:2026-10-03", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 5.824679559744159 - }, - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 5.651770526756559 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "север AND сертификат AND обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 8.32129250405688 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 8.32129250405688 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "север сертификат OR обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 8.32129250405688 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "север сертификат OR обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 8.32129250405688 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "север продление TLS", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "север-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "север продление TLS", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "anyWord", - "score": 2.8258852633782796 - }, - { - "id": "10767", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10762", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10757", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10752", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10747", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10742", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10737", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10732", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10727", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "север-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "север quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "север-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "север quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "anyWord", - "score": 2.8258852633782796 - }, - { - "id": "10767", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10762", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10757", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10752", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10747", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10742", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10737", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10732", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "10727", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "север-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "север сертификат обновление purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "север-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "север сертификат обновление purple", - "relevant": {}, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "anyWord", - "score": 8.47765579013484 - }, - { - "id": "10605", - "chatId": "103", - "match": "anyWord", - "score": 8.32129250405688 - }, - { - "id": "10767", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10762", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10757", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10752", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10747", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10742", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10737", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "10732", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "lumen-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "lumen invocie refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "lumen invocie refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11074", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11069", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11064", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11059", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11054", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11049", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11044", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11039", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11034", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - } - ], - "corrections": [ - { - "from": "invocie", - "to": [ - "invoice" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "lumen invo refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "lumen invo refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11074", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11069", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11064", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11059", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11054", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11049", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11044", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11039", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11034", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"lumen invoice refund\"", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"lumen invoice refund\"", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "lumen invoice from:700", - "relevant": { - "10907": 2 - }, - "hits": [ - { - "id": "10907", - "chatId": "104", - "match": null, - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "lumen invoice from:700", - "relevant": { - "10907": 2 - }, - "hits": [ - { - "id": "10907", - "chatId": "104", - "match": "words", - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "lumen invoice chat:104", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "lumen invoice chat:104", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11199", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11194", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11189", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11184", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11179", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11174", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11169", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11164", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11159", - "chatId": "104", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "lumen invoice date:2026-10-02", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "10908", - "chatId": "104", - "match": null, - "score": 4.8518902533575785 - }, - { - "id": "10907", - "chatId": "104", - "match": null, - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "lumen invoice after:2026-10-02 before:2026-10-03", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "10908", - "chatId": "104", - "match": "words", - "score": 4.8518902533575785 - }, - { - "id": "10907", - "chatId": "104", - "match": "words", - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "lumen AND invoice AND refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "lumen invoice OR refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "lumen invoice OR refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "lumen payment reimbursement", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "lumen payment reimbursement", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11074", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11069", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11064", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11059", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11054", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11049", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11044", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11039", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11034", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "lumen quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "lumen quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11074", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11069", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11064", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11059", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11054", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11049", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11044", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11039", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11034", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "lumen-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "lumen invoice refund purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "lumen invoice refund purple", - "relevant": {}, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11074", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11069", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11064", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11059", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11054", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11049", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11044", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11039", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11034", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "harbor-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "harbor relesae freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "harbor relesae freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11376", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11371", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11366", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11361", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11356", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11351", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11346", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11341", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11336", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - } - ], - "corrections": [ - { - "from": "relesae", - "to": [ - "release" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "harbor rele freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 9.174703389680088 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "harbor rele freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11376", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11371", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11366", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11361", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11356", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11351", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11346", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11341", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11336", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"harbor release freeze\"", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"harbor release freeze\"", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "harbor release from:700", - "relevant": { - "11209": 2 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": null, - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "harbor release from:700", - "relevant": { - "11209": 2 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "words", - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "harbor release chat:105", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "harbor release chat:105", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11501", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11496", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11491", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11486", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11481", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11476", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11471", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11466", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11461", - "chatId": "105", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "harbor release date:2026-10-02", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": null, - "score": 4.8518902533575785 - }, - { - "id": "11210", - "chatId": "105", - "match": null, - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "harbor release after:2026-10-02 before:2026-10-03", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "words", - "score": 4.8518902533575785 - }, - { - "id": "11210", - "chatId": "105", - "match": "words", - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "harbor AND release AND freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "harbor release OR freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "harbor release OR freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "harbor shipping pause", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "harbor shipping pause", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "anyWord", - "score": 10.632628927084141 - }, - { - "id": "11376", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11371", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11366", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11361", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11356", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11351", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11346", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11341", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11336", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [ - { - "from": "pause", - "to": [ - "passes" - ] - } - ], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "harbor quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "harbor quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11376", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11371", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11366", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11361", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11356", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11351", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11346", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11341", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11336", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "harbor-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "harbor release freeze purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "harbor release freeze purple", - "relevant": {}, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11376", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11371", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11366", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11361", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11356", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11351", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11346", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11341", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11336", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "maple-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "maple arhcive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "maple arhcive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11678", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11673", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11668", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11663", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11658", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11653", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11648", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11643", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - }, - { - "id": "11638", - "chatId": "999", - "match": "corrected", - "score": 7.48582930091083 - } - ], - "corrections": [ - { - "from": "arhcive", - "to": [ - "archive" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "maple arch restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "maple arch restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11678", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11673", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11668", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11663", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11658", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11653", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11648", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11643", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - }, - { - "id": "11638", - "chatId": "999", - "match": "beginnings", - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"maple archive restore\"", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"maple archive restore\"", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "maple archive from:700", - "relevant": { - "11511": 2 - }, - "hits": [ - { - "id": "11511", - "chatId": "106", - "match": null, - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "maple archive from:700", - "relevant": { - "11511": 2 - }, - "hits": [ - { - "id": "11511", - "chatId": "106", - "match": "words", - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "maple archive chat:106", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "maple archive chat:106", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11803", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11798", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11793", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11788", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11783", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11778", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11773", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11768", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "11763", - "chatId": "106", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "maple archive date:2026-10-02", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11512", - "chatId": "106", - "match": null, - "score": 4.718335520783594 - }, - { - "id": "11511", - "chatId": "106", - "match": null, - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "maple archive after:2026-10-02 before:2026-10-03", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11512", - "chatId": "106", - "match": "words", - "score": 4.718335520783594 - }, - { - "id": "11511", - "chatId": "106", - "match": "words", - "score": 4.718335520783594 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "maple AND archive AND restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "maple archive OR restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "maple archive OR restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "maple recover saved data", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "maple recover saved data", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11511", - "chatId": "106", - "match": "anyWord", - "score": 10.33995167373274 - }, - { - "id": "11209", - "chatId": "105", - "match": "anyWord", - "score": 8.206683800405353 - }, - { - "id": "11673", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11668", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11663", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11658", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11653", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11648", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11643", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11638", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "maple quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "maple quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11678", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11673", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11668", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11663", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11658", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11653", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11648", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11643", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11638", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "maple-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "maple archive restore purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "maple archive restore purple", - "relevant": {}, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11678", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11673", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11668", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11663", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11658", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11653", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11648", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11643", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - }, - { - "id": "11638", - "chatId": "999", - "match": "anyWord", - "score": 7.48582930091083 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "вектор-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 8.47765579013484 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 8.47765579013484 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "вектор миграця отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - }, - { - "id": "12065", - "chatId": "107", - "match": null, - "score": 10.005267331328891 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "вектор миграця отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "corrected", - "score": 8.47765579013484 - }, - { - "id": "11975", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11970", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11965", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11960", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11955", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11950", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11945", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11940", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "11935", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - } - ], - "corrections": [ - { - "from": "миграця", - "to": [ - "миграция" - ] - } - ], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "вектор мигр отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "вектор мигр отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "beginnings", - "score": 8.47765579013484 - }, - { - "id": "11975", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11970", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11965", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11960", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11955", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11950", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11945", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11940", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "11935", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"вектор миграция отмена\"", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 3.126652600631594 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"вектор миграция отмена\"", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 3.126652600631594 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "вектор миграция from:700", - "relevant": { - "11813": 2 - }, - "hits": [ - { - "id": "11813", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "вектор миграция from:700", - "relevant": { - "11813": 2 - }, - "hits": [ - { - "id": "11813", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "вектор миграция chat:107", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 5.651770526756559 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "вектор миграция chat:107", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 5.651770526756559 - }, - { - "id": "12110", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12105", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12100", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12095", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12090", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12085", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12080", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12075", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12070", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "вектор миграция date:2026-10-02", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 5.651770526756559 - }, - { - "id": "11813", - "chatId": "107", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "вектор миграция after:2026-10-02 before:2026-10-03", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 5.651770526756559 - }, - { - "id": "11813", - "chatId": "107", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "вектор AND миграция AND отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 8.47765579013484 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 8.47765579013484 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "вектор миграция OR отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 8.47765579013484 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "вектор миграция OR отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 8.47765579013484 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "вектор возврат прежней схемы", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "вектор возврат прежней схемы", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11813", - "chatId": "107", - "match": "anyWord", - "score": 10.942357565924837 - }, - { - "id": "11814", - "chatId": "107", - "match": "anyWord", - "score": 2.8258852633782796 - }, - { - "id": "11975", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11970", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11965", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11960", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11955", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11950", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11945", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11940", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [ - { - "from": "схемы", - "to": [ - "схему" - ] - } - ], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "вектор quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "вектор quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "anyWord", - "score": 2.8258852633782796 - }, - { - "id": "11975", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11970", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11965", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11960", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11955", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11950", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11945", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11940", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "11935", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "вектор-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "вектор миграция отмена purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "вектор миграция отмена purple", - "relevant": {}, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "anyWord", - "score": 8.47765579013484 - }, - { - "id": "11975", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11970", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11965", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11960", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11955", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11950", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11945", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11940", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "11935", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "prism-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "prism renweal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "prism renweal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12282", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12277", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12272", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12267", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12262", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12257", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12252", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12247", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - }, - { - "id": "12242", - "chatId": "999", - "match": "corrected", - "score": 7.489838832938161 - } - ], - "corrections": [ - { - "from": "renweal", - "to": [ - "renewal" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "prism rene discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "prism rene discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12282", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12277", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12272", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12267", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12262", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12257", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12252", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12247", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - }, - { - "id": "12242", - "chatId": "999", - "match": "beginnings", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"prism renewal discount\"", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"prism renewal discount\"", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 2.7623348535297314 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "prism renewal from:700", - "relevant": { - "12115": 2 - }, - "hits": [ - { - "id": "12115", - "chatId": "108", - "match": null, - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "prism renewal from:700", - "relevant": { - "12115": 2 - }, - "hits": [ - { - "id": "12115", - "chatId": "108", - "match": "words", - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "prism renewal chat:108", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "prism renewal chat:108", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12407", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12402", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12397", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12392", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12387", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12382", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12377", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12372", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - }, - { - "id": "12367", - "chatId": "108", - "match": "words", - "score": 4.993225888625441 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "prism renewal date:2026-10-02", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12116", - "chatId": "108", - "match": null, - "score": 4.8518902533575785 - }, - { - "id": "12115", - "chatId": "108", - "match": null, - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "prism renewal after:2026-10-02 before:2026-10-03", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12116", - "chatId": "108", - "match": "words", - "score": 4.8518902533575785 - }, - { - "id": "12115", - "chatId": "108", - "match": "words", - "score": 4.8518902533575785 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "prism AND renewal AND discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "prism renewal OR discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "prism renewal OR discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "prism subscription price reduction", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "prism subscription price reduction", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12282", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12277", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12272", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12267", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12262", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12257", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12252", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12247", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - }, - { - "id": "12242", - "chatId": "999", - "match": "anyWord", - "score": 4.993225888625441 - } - ], - "corrections": [ - { - "from": "price", - "to": [ - "prism" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "prism quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "prism quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12282", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12277", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12272", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12267", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12262", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12257", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12252", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12247", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - }, - { - "id": "12242", - "chatId": "999", - "match": "anyWord", - "score": 2.4966129443127203 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "prism-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "prism renewal discount purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "prism renewal discount purple", - "relevant": {}, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12282", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12277", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12272", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12267", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12262", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12257", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12252", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12247", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - }, - { - "id": "12242", - "chatId": "999", - "match": "anyWord", - "score": 7.489838832938161 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - } - ] -} diff --git a/bench/message-search-quality/baseline.json b/bench/message-search-quality/baseline.json deleted file mode 100644 index 2bd5d44ca..000000000 --- a/bench/message-search-quality/baseline.json +++ /dev/null @@ -1,11549 +0,0 @@ -{ - "schemaVersion": 1, - "k": 10, - "node": "v24.19.0", - "timezone": "UTC", - "storePath": "/tmp/message-search-quality-XIyNa5/quality.db", - "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", - "runnerSha256": "ff9cbc4de7a2f6312ba6a2f3565bcda9b6b4105f3c50fc65cbbd7f8ffc1c186c", - "productionSha256": { - "services/messages.js": "62f927cddd912275bb46a7c990d64881c2b4a22eac12d54f484fd5ca4223a3e4", - "services/messages-search.js": "35eb4ed1dcc002a80985327ffa9f6bd25a5ee6b1a31f3024203a38b6605c7d2a", - "search/search.js": "e1c7975b1573936515244fdc60454a699552afde89363b517d35974e115384b6", - "search/query.js": "89a7030d749f06ca61b9867bf29db4244617943e14c383aa5a09fd40ed819772", - "search/correct.js": "54e5089f217840dd9e27e6e89c29486c2d8704850af16dafd1b8410998066270", - "search/stem.js": "84475baa966a17595c480015668dc77da60409a00c391aa9ca6a718433386007", - "search/lucene/parser.js": "28f9a1edbeb172e4556fcc23528c83e2acf008ff70dae48ca99dbcfe62b1531d", - "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964", - "store/sqlite/words.js": "be7c86aa22f8a308ce2fba664801ffd65da4aad444f1a8e62533548980a9e5b3" - }, - "messages": 2616, - "queries": 96, - "metrics": { - "dev": { - "lucene": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3125, - "mrr": 0.325, - "ndcgAt10": 0.2865560469084554, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.25, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.9491768952476267, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - } - } - }, - "legacy": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3875, - "mrr": 0.425, - "ndcgAt10": 0.3539325811955994, - "noAnswerFalseHits": 80, - "noAnswerQueryHitRate": 1, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.25, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.9491768952476267, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.25, - "mrr": 0.5, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - } - } - } - }, - "test": { - "lucene": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.2625, - "mrr": 0.325, - "ndcgAt10": 0.21917951262131133, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.8475306857428799, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": 0 - } - } - }, - "legacy": { - "queries": 48, - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3375, - "mrr": 0.45, - "ndcgAt10": 0.2992618230965487, - "noAnswerFalseHits": 80, - "noAnswerQueryHitRate": 1, - "categories": { - "words": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "typo": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "beginnings": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "phrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "from": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "chat": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "date": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.8475306857428799, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "and": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "or": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.125, - "mrr": 0.25, - "ndcgAt10": 0.06885288809404666, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "paraphrase": { - "queries": 4, - "answerable": 4, - "noAnswerQueries": 0, - "recallAt10": 0.5, - "mrr": 0.75, - "ndcgAt10": 0.66311732856428, - "noAnswerFalseHits": 0, - "noAnswerQueryHitRate": null - }, - "no-answer-absent": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - }, - "no-answer-near-miss": { - "queries": 4, - "answerable": 0, - "noAnswerQueries": 4, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 40, - "noAnswerQueryHitRate": 1 - } - } - } - } - }, - "rankings": [ - { - "id": "aurora-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "aurora rolluot rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "aurora rolluot rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10168", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10163", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10158", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10153", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10148", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10143", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10138", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10133", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10128", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - } - ], - "corrections": [ - { - "from": "rolluot", - "to": [ - "rollout" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "aurora roll rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "aurora roll rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10168", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10163", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10158", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10153", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10148", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10143", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10138", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10133", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10128", - "chatId": "999", - "match": "beginnings", - "score": 7.447603932068213 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"aurora rollout rollback\"", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"aurora rollout rollback\"", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "aurora rollout from:700", - "relevant": { - "10001": 2 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": null, - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "aurora rollout from:700", - "relevant": { - "10001": 2 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": "words", - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "aurora rollout chat:101", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "aurora rollout chat:101", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10293", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10288", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10283", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10278", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10273", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10268", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10263", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10258", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10253", - "chatId": "101", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "aurora rollout date:2026-10-02", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": null, - "score": 4.340227569072966 - }, - { - "id": "10002", - "chatId": "101", - "match": null, - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "aurora rollout after:2026-10-02 before:2026-10-03", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10001", - "chatId": "101", - "match": "words", - "score": 4.340227569072966 - }, - { - "id": "10002", - "chatId": "101", - "match": "words", - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "aurora AND rollout AND rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "aurora rollout OR rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10298", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10293", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10288", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10283", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10278", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10273", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10268", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10263", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10258", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10253", - "chatId": "101", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "aurora rollout OR rollback", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10168", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10163", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10158", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10153", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10148", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10143", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10138", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10133", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10128", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "aurora deployment reversal", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "aurora deployment reversal", - "relevant": { - "10001": 2, - "10002": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "anyWord", - "score": 7.95981823822498 - }, - { - "id": "10168", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10163", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10158", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10153", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10148", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10143", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10138", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10133", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10128", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "aurora quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "aurora quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10168", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10163", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10158", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10153", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10148", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10143", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10138", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10133", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10128", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "aurora-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "aurora rollout rollback purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "aurora-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "aurora rollout rollback purple", - "relevant": {}, - "hits": [ - { - "id": "10173", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10168", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10163", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10158", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10153", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10148", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10143", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10138", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10133", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10128", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "cedar-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "cedar bakcup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "cedar bakcup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10465", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10460", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10455", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10450", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10445", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10440", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10435", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10430", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10425", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - } - ], - "corrections": [ - { - "from": "bakcup", - "to": [ - "backup" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "cedar back retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "cedar back retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10465", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10460", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10455", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10450", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10445", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10440", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10435", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10430", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10425", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"cedar backup retention\"", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"cedar backup retention\"", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "cedar backup from:700", - "relevant": { - "10303": 2 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "cedar backup from:700", - "relevant": { - "10303": 2 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "cedar backup chat:102", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "cedar backup chat:102", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10595", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10590", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10585", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10580", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10575", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10570", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10565", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10560", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10555", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "cedar backup date:2026-10-02", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10304", - "chatId": "102", - "match": null, - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "cedar backup after:2026-10-02 before:2026-10-03", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10304", - "chatId": "102", - "match": "words", - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "cedar AND backup AND retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "cedar backup OR retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10600", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10595", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10590", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10585", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10580", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10575", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10570", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10565", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10560", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10555", - "chatId": "102", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "cedar backup OR retention", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10465", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10460", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10455", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10450", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10445", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10440", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10435", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10430", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10425", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "cedar snapshot lifetime", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "cedar snapshot lifetime", - "relevant": { - "10303": 2, - "10304": 1 - }, - "hits": [ - { - "id": "10303", - "chatId": "102", - "match": "anyWord", - "score": 9.865391899469309 - }, - { - "id": "11511", - "chatId": "106", - "match": "anyWord", - "score": 7.209540247771293 - }, - { - "id": "10465", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10460", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10455", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10450", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10445", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10440", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10435", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10430", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "cedar quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "cedar quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10465", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10460", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10455", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10450", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10445", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10440", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10435", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10430", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10425", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "cedar-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "cedar backup retention purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "cedar-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "cedar backup retention purple", - "relevant": {}, - "hits": [ - { - "id": "10470", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10465", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10460", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10455", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10450", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10445", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10440", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10435", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10430", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10425", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "север-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 7.447603932068213 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 7.447603932068213 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "север сертифкат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "север-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "север сертифкат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "corrected", - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": "corrected", - "score": 7.447603932068213 - }, - { - "id": "10767", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10762", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10757", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10752", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10747", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10742", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10737", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "10732", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - } - ], - "corrections": [ - { - "from": "сертифкат", - "to": [ - "сертификат" - ] - } - ], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "север серти обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "север-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "север серти обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "beginnings", - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": "beginnings", - "score": 7.447603932068213 - }, - { - "id": "10767", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10762", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10757", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10752", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10747", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10742", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10737", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "10732", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"север сертификат обновление\"", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 2.8366598529202736 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "10862", - "chatId": "103", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "север-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"север сертификат обновление\"", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 2.8366598529202736 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "10727", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "север-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "север сертификат from:700", - "relevant": { - "10605": 2 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 5.213948397197031 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "север сертификат from:700", - "relevant": { - "10605": 2 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 5.213948397197031 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "север сертификат chat:103", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 5.213948397197031 - }, - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 5.059938729655221 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "север сертификат chat:103", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 5.213948397197031 - }, - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 5.059938729655221 - }, - { - "id": "10902", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10897", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10892", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10887", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10882", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10877", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10872", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "10867", - "chatId": "103", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "север сертификат date:2026-10-02", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 5.213948397197031 - }, - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 5.059938729655221 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "север сертификат after:2026-10-02 before:2026-10-03", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 5.213948397197031 - }, - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 5.059938729655221 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "север-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "север AND сертификат AND обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 7.447603932068213 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 7.447603932068213 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "север сертификат OR обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": null, - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": null, - "score": 7.447603932068213 - }, - { - "id": "10902", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10897", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10892", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10887", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10882", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10877", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10872", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "10867", - "chatId": "103", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "север сертификат OR обновление", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "words", - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": "words", - "score": 7.447603932068213 - }, - { - "id": "10767", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10762", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10757", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10752", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10747", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10742", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10737", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "10732", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "север-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "север продление TLS", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "север-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "север продление TLS", - "relevant": { - "10605": 2, - "10606": 1 - }, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "anyWord", - "score": 2.5299693648276107 - }, - { - "id": "10767", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10762", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10757", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10752", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10747", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10742", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10737", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10732", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10727", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "север-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "север quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "север-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "север quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "anyWord", - "score": 2.5299693648276107 - }, - { - "id": "10767", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10762", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10757", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10752", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10747", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10742", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10737", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10732", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "10727", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "север-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "север сертификат обновление purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "север-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "север сертификат обновление purple", - "relevant": {}, - "hits": [ - { - "id": "10606", - "chatId": "103", - "match": "anyWord", - "score": 7.589908094482832 - }, - { - "id": "10605", - "chatId": "103", - "match": "anyWord", - "score": 7.447603932068213 - }, - { - "id": "10767", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10762", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10757", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10752", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10747", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10742", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10737", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "10732", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "lumen-words", - "split": "dev", - "category": "words", - "mode": "lucene", - "text": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-words", - "split": "dev", - "category": "words", - "mode": "legacy", - "text": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-typo", - "split": "dev", - "category": "typo", - "mode": "lucene", - "text": "lumen invocie refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-typo", - "split": "dev", - "category": "typo", - "mode": "legacy", - "text": "lumen invocie refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11074", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11069", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11064", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11059", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11054", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11049", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11044", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11039", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11034", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - } - ], - "corrections": [ - { - "from": "invocie", - "to": [ - "invoice" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "lucene", - "text": "lumen invo refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-beginnings", - "split": "dev", - "category": "beginnings", - "mode": "legacy", - "text": "lumen invo refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11074", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11069", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11064", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11059", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11054", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11049", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11044", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11039", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11034", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-phrase", - "split": "dev", - "category": "phrase", - "mode": "lucene", - "text": "\"lumen invoice refund\"", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-phrase", - "split": "dev", - "category": "phrase", - "mode": "legacy", - "text": "\"lumen invoice refund\"", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-from", - "split": "dev", - "category": "from", - "mode": "lucene", - "text": "lumen invoice from:700", - "relevant": { - "10907": 2 - }, - "hits": [ - { - "id": "10907", - "chatId": "104", - "match": null, - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-from", - "split": "dev", - "category": "from", - "mode": "legacy", - "text": "lumen invoice from:700", - "relevant": { - "10907": 2 - }, - "hits": [ - { - "id": "10907", - "chatId": "104", - "match": "words", - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-chat", - "split": "dev", - "category": "chat", - "mode": "lucene", - "text": "lumen invoice chat:104", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-chat", - "split": "dev", - "category": "chat", - "mode": "legacy", - "text": "lumen invoice chat:104", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11199", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11194", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11189", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11184", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11179", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11174", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11169", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11164", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11159", - "chatId": "104", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-date", - "split": "dev", - "category": "date", - "mode": "lucene", - "text": "lumen invoice date:2026-10-02", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "10908", - "chatId": "104", - "match": null, - "score": 4.340227569072966 - }, - { - "id": "10907", - "chatId": "104", - "match": null, - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-date", - "split": "dev", - "category": "date", - "mode": "legacy", - "text": "lumen invoice after:2026-10-02 before:2026-10-03", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "10908", - "chatId": "104", - "match": "words", - "score": 4.340227569072966 - }, - { - "id": "10907", - "chatId": "104", - "match": "words", - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-and", - "split": "dev", - "category": "and", - "mode": "lucene", - "text": "lumen AND invoice AND refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-and", - "split": "dev", - "category": "and", - "mode": "legacy", - "text": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-or", - "split": "dev", - "category": "or", - "mode": "lucene", - "text": "lumen invoice OR refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11204", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11199", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11194", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11189", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11184", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11179", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11174", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11169", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11164", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11159", - "chatId": "104", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-or", - "split": "dev", - "category": "or", - "mode": "legacy", - "text": "lumen invoice OR refund", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11074", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11069", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11064", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11059", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11054", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11049", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11044", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11039", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11034", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "lucene", - "text": "lumen payment reimbursement", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-paraphrase", - "split": "dev", - "category": "paraphrase", - "mode": "legacy", - "text": "lumen payment reimbursement", - "relevant": { - "10907": 2, - "10908": 1 - }, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11074", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11069", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11064", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11059", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11054", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11049", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11044", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11039", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11034", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "lucene", - "text": "lumen quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-absent", - "split": "dev", - "category": "no-answer-absent", - "mode": "legacy", - "text": "lumen quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11074", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11069", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11064", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11059", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11054", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11049", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11044", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11039", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11034", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "lumen-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "lumen invoice refund purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "lumen-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "lumen invoice refund purple", - "relevant": {}, - "hits": [ - { - "id": "11079", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11074", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11069", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11064", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11059", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11054", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11049", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11044", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11039", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11034", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "harbor-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "harbor relesae freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "harbor relesae freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11376", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11371", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11366", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11361", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11356", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11351", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11346", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11341", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11336", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - } - ], - "corrections": [ - { - "from": "relesae", - "to": [ - "release" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "harbor rele freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 8.213731853767285 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "harbor rele freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11376", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11371", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11366", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11361", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11356", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11351", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11346", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11341", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11336", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"harbor release freeze\"", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"harbor release freeze\"", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "harbor release from:700", - "relevant": { - "11209": 2 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": null, - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "harbor release from:700", - "relevant": { - "11209": 2 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "words", - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "harbor release chat:105", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "harbor release chat:105", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11501", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11496", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11491", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11486", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11481", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11476", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11471", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11466", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11461", - "chatId": "105", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "harbor release date:2026-10-02", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": null, - "score": 4.340227569072966 - }, - { - "id": "11210", - "chatId": "105", - "match": null, - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "harbor release after:2026-10-02 before:2026-10-03", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "words", - "score": 4.340227569072966 - }, - { - "id": "11210", - "chatId": "105", - "match": "words", - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "harbor AND release AND freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "harbor release OR freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11501", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11496", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11491", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11486", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11481", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11476", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11471", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11466", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "11461", - "chatId": "105", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "harbor release OR freeze", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11376", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11371", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11366", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11361", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11356", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11351", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11346", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11341", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11336", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "harbor shipping pause", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "harbor shipping pause", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "chatId": "105", - "match": "anyWord", - "score": 10.129932022761462 - }, - { - "id": "11376", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11371", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11366", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11361", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11356", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11351", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11346", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11341", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11336", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [ - { - "from": "pause", - "to": [ - "passes" - ] - } - ], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "harbor quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "harbor quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11376", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11371", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11366", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11361", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11356", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11351", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11346", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11341", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11336", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "harbor-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "harbor release freeze purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "harbor-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "harbor release freeze purple", - "relevant": {}, - "hits": [ - { - "id": "11381", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11376", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11371", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11366", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11361", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11356", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11351", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11346", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11341", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11336", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "maple-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "maple arhcive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "maple arhcive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11678", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11673", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11668", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11663", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11658", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11653", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11648", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11643", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - }, - { - "id": "11638", - "chatId": "999", - "match": "corrected", - "score": 6.696869489999996 - } - ], - "corrections": [ - { - "from": "arhcive", - "to": [ - "archive" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "maple arch restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "maple arch restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11678", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11673", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11668", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11663", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11658", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11653", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11648", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11643", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - }, - { - "id": "11638", - "chatId": "999", - "match": "beginnings", - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"maple archive restore\"", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"maple archive restore\"", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "maple archive from:700", - "relevant": { - "11511": 2 - }, - "hits": [ - { - "id": "11511", - "chatId": "106", - "match": null, - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "maple archive from:700", - "relevant": { - "11511": 2 - }, - "hits": [ - { - "id": "11511", - "chatId": "106", - "match": "words", - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "maple archive chat:106", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "maple archive chat:106", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11803", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11798", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11793", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11788", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11783", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11778", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11773", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11768", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "11763", - "chatId": "106", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "maple archive date:2026-10-02", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11512", - "chatId": "106", - "match": null, - "score": 4.220174468555267 - }, - { - "id": "11511", - "chatId": "106", - "match": null, - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "maple archive after:2026-10-02 before:2026-10-03", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11512", - "chatId": "106", - "match": "words", - "score": 4.220174468555267 - }, - { - "id": "11511", - "chatId": "106", - "match": "words", - "score": 4.220174468555267 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "maple AND archive AND restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "maple archive OR restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11803", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11798", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11793", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11788", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11783", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11778", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11773", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11768", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - }, - { - "id": "11763", - "chatId": "106", - "match": null, - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "maple archive OR restore", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11678", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11673", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11668", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11663", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11658", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11653", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11648", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11643", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - }, - { - "id": "11638", - "chatId": "999", - "match": "words", - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "maple recover saved data", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "maple recover saved data", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11511", - "chatId": "106", - "match": "anyWord", - "score": 9.849732487596121 - }, - { - "id": "11209", - "chatId": "105", - "match": "anyWord", - "score": 7.95981823822498 - }, - { - "id": "11673", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11668", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11663", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11658", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11653", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11648", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11643", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11638", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "maple quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "maple quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11678", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11673", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11668", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11663", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11658", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11653", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11648", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11643", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11638", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "maple-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "maple archive restore purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "maple-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "maple archive restore purple", - "relevant": {}, - "hits": [ - { - "id": "11683", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11678", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11673", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11668", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11663", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11658", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11653", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11648", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11643", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - }, - { - "id": "11638", - "chatId": "999", - "match": "anyWord", - "score": 6.696869489999996 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "вектор-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 7.589908094482832 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 7.589908094482832 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "вектор миграця отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - }, - { - "id": "12065", - "chatId": "107", - "match": null, - "score": 8.959591949608743 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "вектор миграця отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "corrected", - "score": 7.589908094482832 - }, - { - "id": "11975", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11970", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11965", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11960", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11955", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11950", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11945", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11940", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "11935", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - } - ], - "corrections": [ - { - "from": "миграця", - "to": [ - "миграция" - ] - } - ], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "вектор мигр отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "вектор мигр отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "beginnings", - "score": 7.589908094482832 - }, - { - "id": "11975", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11970", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11965", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11960", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11955", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11950", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11945", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11940", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "11935", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"вектор миграция отмена\"", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 2.8366598529202736 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"вектор миграция отмена\"", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 2.8366598529202736 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "вектор миграция from:700", - "relevant": { - "11813": 2 - }, - "hits": [ - { - "id": "11813", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "вектор миграция from:700", - "relevant": { - "11813": 2 - }, - "hits": [ - { - "id": "11813", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "вектор миграция chat:107", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 5.059938729655221 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "вектор миграция chat:107", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 5.059938729655221 - }, - { - "id": "12110", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12105", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12100", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12095", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12090", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12085", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12080", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12075", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12070", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "вектор миграция date:2026-10-02", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 5.059938729655221 - }, - { - "id": "11813", - "chatId": "107", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "вектор миграция after:2026-10-02 before:2026-10-03", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 5.059938729655221 - }, - { - "id": "11813", - "chatId": "107", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "вектор AND миграция AND отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 7.589908094482832 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 7.589908094482832 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "вектор миграция OR отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": null, - "score": 7.589908094482832 - }, - { - "id": "12110", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12105", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12100", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12095", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12090", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12085", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12080", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12075", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12070", - "chatId": "107", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "вектор миграция OR отмена", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "words", - "score": 7.589908094482832 - }, - { - "id": "11975", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11970", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11965", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11960", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11955", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11950", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11945", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11940", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "11935", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "вектор возврат прежней схемы", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "вектор возврат прежней схемы", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11813", - "chatId": "107", - "match": "anyWord", - "score": 10.426540254129044 - }, - { - "id": "11814", - "chatId": "107", - "match": "anyWord", - "score": 2.5299693648276107 - }, - { - "id": "11975", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11970", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11965", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11960", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11955", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11950", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11945", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11940", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [ - { - "from": "схемы", - "to": [ - "схему" - ] - } - ], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "вектор quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "вектор quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "anyWord", - "score": 2.5299693648276107 - }, - { - "id": "11975", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11970", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11965", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11960", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11955", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11950", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11945", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11940", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "11935", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "вектор-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "вектор миграция отмена purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "вектор-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "вектор миграция отмена purple", - "relevant": {}, - "hits": [ - { - "id": "11814", - "chatId": "107", - "match": "anyWord", - "score": 7.589908094482832 - }, - { - "id": "11975", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11970", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11965", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11960", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11955", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11950", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11945", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11940", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "11935", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "prism-words", - "split": "test", - "category": "words", - "mode": "lucene", - "text": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-words", - "split": "test", - "category": "words", - "mode": "legacy", - "text": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-typo", - "split": "test", - "category": "typo", - "mode": "lucene", - "text": "prism renweal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-typo", - "split": "test", - "category": "typo", - "mode": "legacy", - "text": "prism renweal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12282", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12277", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12272", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12267", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12262", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12257", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12252", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12247", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - }, - { - "id": "12242", - "chatId": "999", - "match": "corrected", - "score": 6.700966604613545 - } - ], - "corrections": [ - { - "from": "renweal", - "to": [ - "renewal" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-beginnings", - "split": "test", - "category": "beginnings", - "mode": "lucene", - "text": "prism rene discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-beginnings", - "split": "test", - "category": "beginnings", - "mode": "legacy", - "text": "prism rene discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12282", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12277", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12272", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12267", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12262", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12257", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12252", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12247", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - }, - { - "id": "12242", - "chatId": "999", - "match": "beginnings", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-phrase", - "split": "test", - "category": "phrase", - "mode": "lucene", - "text": "\"prism renewal discount\"", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-phrase", - "split": "test", - "category": "phrase", - "mode": "legacy", - "text": "\"prism renewal discount\"", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 2.5044259701753258 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-from", - "split": "test", - "category": "from", - "mode": "lucene", - "text": "prism renewal from:700", - "relevant": { - "12115": 2 - }, - "hits": [ - { - "id": "12115", - "chatId": "108", - "match": null, - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-from", - "split": "test", - "category": "from", - "mode": "legacy", - "text": "prism renewal from:700", - "relevant": { - "12115": 2 - }, - "hits": [ - { - "id": "12115", - "chatId": "108", - "match": "words", - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-chat", - "split": "test", - "category": "chat", - "mode": "lucene", - "text": "prism renewal chat:108", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-chat", - "split": "test", - "category": "chat", - "mode": "legacy", - "text": "prism renewal chat:108", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12407", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12402", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12397", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12392", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12387", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12382", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12377", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12372", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - }, - { - "id": "12367", - "chatId": "108", - "match": "words", - "score": 4.4673110697423635 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-date", - "split": "test", - "category": "date", - "mode": "lucene", - "text": "prism renewal date:2026-10-02", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12116", - "chatId": "108", - "match": null, - "score": 4.340227569072966 - }, - { - "id": "12115", - "chatId": "108", - "match": null, - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-date", - "split": "test", - "category": "date", - "mode": "legacy", - "text": "prism renewal after:2026-10-02 before:2026-10-03", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12116", - "chatId": "108", - "match": "words", - "score": 4.340227569072966 - }, - { - "id": "12115", - "chatId": "108", - "match": "words", - "score": 4.340227569072966 - } - ], - "corrections": [], - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-and", - "split": "test", - "category": "and", - "mode": "lucene", - "text": "prism AND renewal AND discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-and", - "split": "test", - "category": "and", - "mode": "legacy", - "text": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-or", - "split": "test", - "category": "or", - "mode": "lucene", - "text": "prism renewal OR discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12407", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12402", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12397", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12392", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12387", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12382", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12377", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12372", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - }, - { - "id": "12367", - "chatId": "108", - "match": null, - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-or", - "split": "test", - "category": "or", - "mode": "legacy", - "text": "prism renewal OR discount", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12282", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12277", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12272", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12267", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12262", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12257", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12252", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12247", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - }, - { - "id": "12242", - "chatId": "999", - "match": "words", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "lucene", - "text": "prism subscription price reduction", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [], - "corrections": [], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-paraphrase", - "split": "test", - "category": "paraphrase", - "mode": "legacy", - "text": "prism subscription price reduction", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12282", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12277", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12272", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12267", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12262", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12257", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12252", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12247", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - }, - { - "id": "12242", - "chatId": "999", - "match": "anyWord", - "score": 4.4673110697423635 - } - ], - "corrections": [ - { - "from": "price", - "to": [ - "prism" - ] - } - ], - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "lucene", - "text": "prism quantum unicorn", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-absent", - "split": "test", - "category": "no-answer-absent", - "mode": "legacy", - "text": "prism quantum unicorn", - "relevant": {}, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12282", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12277", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12272", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12267", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12262", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12257", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12252", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12247", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - }, - { - "id": "12242", - "chatId": "999", - "match": "anyWord", - "score": 2.2336555348711817 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - }, - { - "id": "prism-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "lucene", - "text": "prism renewal discount purple", - "relevant": {}, - "hits": [], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 0 - }, - { - "id": "prism-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "mode": "legacy", - "text": "prism renewal discount purple", - "relevant": {}, - "hits": [ - { - "id": "12287", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12282", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12277", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12272", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12267", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12262", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12257", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12252", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12247", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - }, - { - "id": "12242", - "chatId": "999", - "match": "anyWord", - "score": 6.700966604613545 - } - ], - "corrections": [], - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "noAnswerFalseHits": 10 - } - ] -} diff --git a/bench/message-search-quality/combined-dev.json b/bench/message-search-quality/combined-dev.json deleted file mode 100644 index 7832edb41..000000000 --- a/bench/message-search-quality/combined-dev.json +++ /dev/null @@ -1,3723 +0,0 @@ -{ - "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", - "productionSha256": { - "services/messages-combined.js": "d10aeb5f7049f615503d59c37df109bcd284f53b5cfceaac0151f2d7e2ca60e9", - "services/messages-search.js": "35eb4ed1dcc002a80985327ffa9f6bd25a5ee6b1a31f3024203a38b6605c7d2a", - "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", - "search/correct.js": "54e5089f217840dd9e27e6e89c29486c2d8704850af16dafd1b8410998066270", - "search/stem.js": "84475baa966a17595c480015668dc77da60409a00c391aa9ca6a718433386007", - "store/normalize.js": "2c59c9a7c7486b37754b639d26d036c20054aff2c18470ba0524115d9da24de0", - "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964" - }, - "runnerSha256": "b71dd53f4ba99a5d690a313569b048c0bc801d27fa29576e74ec0b58fa50ee44", - "configurations": [ - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0989270000000033, - "p95Ms": 5.586588999999947 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.820695999999998, - "p95Ms": 4.736476000000039 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.999600999999984, - "p95Ms": 8.122931999999992 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0147190000000137, - "p95Ms": 5.35894200000007 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0481620000000476, - "p95Ms": 5.102151000000049 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.9879390000000967, - "p95Ms": 4.4711580000000595 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1766829999999118, - "p95Ms": 5.639487000000145 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.120296999999937, - "p95Ms": 4.803971999999931 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1359469999999874, - "p95Ms": 4.584931999999981 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.931483000000071, - "p95Ms": 4.990330999999969 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.067167999999583, - "p95Ms": 5.065161999999873 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.195528000000195, - "p95Ms": 5.348733000000266 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.9227169999999205, - "p95Ms": 4.651435999999649 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0846299999998337, - "p95Ms": 4.821284000000105 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.550333000000137, - "p95Ms": 6.034327000000303 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.6591629999998077, - "p95Ms": 4.640214999999898 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.909552000000076, - "p95Ms": 5.487913999999819 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.993449999999939, - "p95Ms": 5.075561000000107 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.9848830000000817, - "p95Ms": 4.624846000000161 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1461449999997058, - "p95Ms": 5.130724999999984 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.8890740000001642, - "p95Ms": 4.975163000000066 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1842569999998886, - "p95Ms": 4.559173000000101 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.094759000000522, - "p95Ms": 4.604597999999896 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.8531670000002123, - "p95Ms": 4.757614999999532 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1271700000006604, - "p95Ms": 4.51660400000037 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2620529999994687, - "p95Ms": 5.642542999999932 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.9432660000002215, - "p95Ms": 5.173072999999931 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0052319999995234, - "p95Ms": 4.6353769999996075 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2169080000003305, - "p95Ms": 4.748657999999523 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 60, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.071444999999585, - "p95Ms": 4.6720350000005055 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.825594999999339, - "p95Ms": 4.481768000000557 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.948145000000295, - "p95Ms": 5.107079999999769 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.072377000000415, - "p95Ms": 4.858803999999509 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.141276999999718, - "p95Ms": 4.986815000000206 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1549220000006244, - "p95Ms": 4.732707999999548 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.8966380000001664, - "p95Ms": 4.789594999999281 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.083798999999999, - "p95Ms": 4.598597000000154 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1582180000004882, - "p95Ms": 4.535899999999856 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.871831999999813, - "p95Ms": 4.872200000000703 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1740379999992, - "p95Ms": 4.445510999999897 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1011509999998452, - "p95Ms": 4.676784000000225 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.089318999999705, - "p95Ms": 4.93543899999986 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2209860000002664, - "p95Ms": 5.816338999999971 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0475500000002285, - "p95Ms": 4.833337000000029 - } - }, - { - "options": { - "candidateDepth": 50, - "rrfK": 100, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.9232780000002094, - "p95Ms": 4.709133999999722 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1602720000000772, - "p95Ms": 5.875841000000037 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.964536000001317, - "p95Ms": 5.311302000000069 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.943435999999565, - "p95Ms": 5.634228000000803 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3664279999993596, - "p95Ms": 5.946183000000019 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3577520000017103, - "p95Ms": 5.880669000000125 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4458269999995537, - "p95Ms": 5.950079000000187 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4552650000005087, - "p95Ms": 6.402388000000428 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3393580000010843, - "p95Ms": 5.944709999999759 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.42479800000001, - "p95Ms": 6.081726000000344 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.377369000001636, - "p95Ms": 5.775934000001143 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4498050000001967, - "p95Ms": 5.79357600000003 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3980179999998654, - "p95Ms": 6.328649000000951 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3770779999995284, - "p95Ms": 5.879417000000103 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3765569999995932, - "p95Ms": 5.947375000001557 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 20, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.438093000000663, - "p95Ms": 6.328408000001218 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.979233000000022, - "p95Ms": 5.716100999999981 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.99686500000098, - "p95Ms": 5.263794000000416 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.9873480000005657, - "p95Ms": 5.555910999999469 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.24366800000098, - "p95Ms": 7.042266000000382 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.120136999999886, - "p95Ms": 5.631272000000536 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1539000000011583, - "p95Ms": 7.556660000000193 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2382180000004155, - "p95Ms": 6.014180000000124 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2509419999987585, - "p95Ms": 6.167507000000114 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3813960000006773, - "p95Ms": 6.469952999999805 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.154620999999679, - "p95Ms": 6.012516999999207 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4117029999997612, - "p95Ms": 9.632669999999052 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2852060000004712, - "p95Ms": 6.46590599999945 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2030119999999442, - "p95Ms": 6.237648000000263 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.281227999999828, - "p95Ms": 5.967381999998906 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 60, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2434680000005756, - "p95Ms": 6.523754999998346 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.147037000000637, - "p95Ms": 5.403916000001118 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 2.8970179999996617, - "p95Ms": 6.438083999999435 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.028715000000375, - "p95Ms": 5.09686099999999 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.493426000000909, - "p95Ms": 6.166385000000446 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3336959999996907, - "p95Ms": 6.222479999998541 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.337985000000117, - "p95Ms": 5.702556000000186 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.463880999999674, - "p95Ms": 6.249671000001399 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4468889999989187, - "p95Ms": 5.7435329999989335 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.197732000000542, - "p95Ms": 6.895350999999209 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4615969999995286, - "p95Ms": 10.654305000000022 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3023880000000645, - "p95Ms": 5.465963000000556 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.134103000000323, - "p95Ms": 6.911190000000715 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1532289999995555, - "p95Ms": 5.701062999998612 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2860579999996844, - "p95Ms": 6.078510000001188 - } - }, - { - "options": { - "candidateDepth": 100, - "rrfK": 100, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.207480000000942, - "p95Ms": 7.317231000000902 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.100669999999809, - "p95Ms": 9.202213999997184 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2173079999993206, - "p95Ms": 7.690781000001152 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1045169999997597, - "p95Ms": 9.821302999996988 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3173769999993965, - "p95Ms": 10.891920000001846 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.295234999997774, - "p95Ms": 11.62250899999708 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.147939000002225, - "p95Ms": 9.846882000001642 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.125597000002017, - "p95Ms": 10.109012999997503 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1926320000020496, - "p95Ms": 10.076131999998324 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.145183000000543, - "p95Ms": 10.246551000000181 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1659529999997176, - "p95Ms": 10.46642200000133 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.196890999999596, - "p95Ms": 10.91766799999823 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2881820000002335, - "p95Ms": 11.291237999997975 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.236605000001873, - "p95Ms": 9.723580999998376 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.333716999997705, - "p95Ms": 11.087255999998888 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 20, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.184436999999889, - "p95Ms": 10.429353000003175 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0512780000026396, - "p95Ms": 8.476332999998704 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.172945999998774, - "p95Ms": 7.6514379999971425 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0637710000009974, - "p95Ms": 9.070777000000817 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.162156000002142, - "p95Ms": 10.658501999998407 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.143499999998312, - "p95Ms": 10.011179999997694 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.213631999999052, - "p95Ms": 10.287045999997645 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3347690000009607, - "p95Ms": 10.174446000000898 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2254939999984344, - "p95Ms": 10.048009000001912 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2880409999997937, - "p95Ms": 12.38845399999991 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.256471999997302, - "p95Ms": 12.288046000001486 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.211076999999932, - "p95Ms": 10.37240699999893 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1490610000000743, - "p95Ms": 9.428126999999222 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.7868760000019392, - "p95Ms": 10.107739999999467 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.550893999999971, - "p95Ms": 9.652497000002768 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 60, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0478020000009565, - "p95Ms": 14.126581999997143 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0947699999996985, - "p95Ms": 7.647890999996889 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.140835000001971, - "p95Ms": 7.737259000001359 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.240011999998387, - "p95Ms": 10.344193999997515 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 4.588929999998072, - "p95Ms": 13.008075999998255 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2248130000007222, - "p95Ms": 8.985486999998102 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.169378999999026, - "p95Ms": 9.119418000002042 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.137829999999667, - "p95Ms": 11.149172000001272 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 4.387360999997327, - "p95Ms": 10.366034999999101 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1787359999980254, - "p95Ms": 9.140016999997897 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0379629999988538, - "p95Ms": 12.552051000002393 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1403140000002168, - "p95Ms": 9.01670699999886 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.640871999999945, - "p95Ms": 11.654589000001579 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.325831999998627, - "p95Ms": 9.39201999999932 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3790619999999763, - "p95Ms": 9.77812300000005 - } - }, - { - "options": { - "candidateDepth": 300, - "rrfK": 100, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.178325999997469, - "p95Ms": 10.559667999998055 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1289729999989504, - "p95Ms": 8.623279000003095 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0909520000022894, - "p95Ms": 8.486472000000504 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.165442000001349, - "p95Ms": 8.397134999999253 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.136797999999544, - "p95Ms": 9.770067999997991 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2483370000009018, - "p95Ms": 10.034593999997014 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3170260000006238, - "p95Ms": 11.129775999997946 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3663180000003194, - "p95Ms": 10.832407999998395 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.340158999999403, - "p95Ms": 11.217831000001752 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1947459999973944, - "p95Ms": 10.823953000002803 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.260519999999815, - "p95Ms": 10.102961000000505 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2751270000007935, - "p95Ms": 10.182721000001038 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1247150000017427, - "p95Ms": 10.908601999999519 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2969080000002577, - "p95Ms": 10.397753999997803 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.223972000003414, - "p95Ms": 10.098493000001326 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 20, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.16842700000052, - "p95Ms": 11.122252000001026 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3580820000024687, - "p95Ms": 10.930592000000615 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2287099999994098, - "p95Ms": 8.901921000000584 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.134644000001572, - "p95Ms": 8.8235339999992 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1865209999996296, - "p95Ms": 11.053323000000091 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.318026999997528, - "p95Ms": 10.279723000003287 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.210897000000841, - "p95Ms": 10.048931000001176 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1660120000015013, - "p95Ms": 10.784888999998657 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2241810000014084, - "p95Ms": 10.18936400000166 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2118379999992612, - "p95Ms": 10.870098999999755 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2974390000017593, - "p95Ms": 12.574764000000869 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2604190000020026, - "p95Ms": 10.03881100000217 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.124244999999064, - "p95Ms": 9.719203000000562 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.260980000006384, - "p95Ms": 9.93393400000059 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1301149999999325, - "p95Ms": 10.841444999998203 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 60, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.223489999996673, - "p95Ms": 10.250748999998905 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.157636999996612, - "p95Ms": 8.342072000006738 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "none", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.19188099999883, - "p95Ms": 8.674554999997781 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "none", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.102082999997947, - "p95Ms": 8.305793999999878 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2199129999935394, - "p95Ms": 12.216662999999244 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 4.07511599999998, - "p95Ms": 19.6373969999986 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "coverage", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.1366780000025756, - "p95Ms": 10.087242000001424 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.272711999998137, - "p95Ms": 11.465043999996851 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4137569999948028, - "p95Ms": 9.849736999996821 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "proximity", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4071749999930034, - "p95Ms": 10.754502999996475 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.142929000001459, - "p95Ms": 10.303015999998024 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.5304660000037984, - "p95Ms": 9.76239400000486 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "phrase", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4685190000018338, - "p95Ms": 11.494158000001335 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "all", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.4173440000013215, - "p95Ms": 15.484094999999797 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "all", - "chatCap": 3 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.2240810000002966, - "p95Ms": 10.975125999997545 - } - }, - { - "options": { - "candidateDepth": 500, - "rrfK": 100, - "rerank": "all", - "chatCap": 5 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.3575010000058683, - "p95Ms": 10.992918999996618 - } - } - ], - "devPassed": false, - "chosen": { - "options": { - "candidateDepth": 50, - "rrfK": 20, - "rerank": "none", - "chatCap": 0 - }, - "metrics": { - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0, - "p50Ms": 3.0989270000000033, - "p95Ms": 5.586588999999947 - } - }, - "devRankings": [ - { - "id": "aurora-words", - "category": "words", - "ids": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253" - ], - "elapsedMs": 3.976211000001058, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-typo", - "category": "typo", - "ids": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253" - ], - "elapsedMs": 4.017217000000528, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "ids": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253" - ], - "elapsedMs": 2.7637489999979152, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-phrase", - "category": "phrase", - "ids": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253" - ], - "elapsedMs": 1.8199510000049486, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-from", - "category": "from", - "ids": [ - "10001" - ], - "elapsedMs": 3.2671019999979762, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "aurora-chat", - "category": "chat", - "ids": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253" - ], - "elapsedMs": 3.378069999998843, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-date", - "category": "date", - "ids": [ - "10001", - "10002" - ], - "elapsedMs": 3.196129000003566, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "aurora-and", - "category": "and", - "ids": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253" - ], - "elapsedMs": 1.734269000000495, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-or", - "category": "or", - "ids": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253" - ], - "elapsedMs": 1.7011679999995977, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "ids": [], - "elapsedMs": 2.1651070000007167, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "ids": [], - "elapsedMs": 2.0100359999996726, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "ids": [], - "elapsedMs": 2.2358600000006845, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - { - "id": "cedar-words", - "category": "words", - "ids": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555" - ], - "elapsedMs": 3.978183999999601, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-typo", - "category": "typo", - "ids": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555" - ], - "elapsedMs": 3.918993999999657, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "ids": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555" - ], - "elapsedMs": 2.6834280000039143, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-phrase", - "category": "phrase", - "ids": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555" - ], - "elapsedMs": 1.6508929999981774, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-from", - "category": "from", - "ids": [ - "10303" - ], - "elapsedMs": 3.0946489999987534, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "cedar-chat", - "category": "chat", - "ids": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555" - ], - "elapsedMs": 3.4562460000015562, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-date", - "category": "date", - "ids": [ - "10303", - "10304" - ], - "elapsedMs": 3.1982330000028014, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "cedar-and", - "category": "and", - "ids": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555" - ], - "elapsedMs": 1.737676000004285, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-or", - "category": "or", - "ids": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555" - ], - "elapsedMs": 1.6936129999958212, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "ids": [], - "elapsedMs": 1.9198580000011134, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "ids": [], - "elapsedMs": 1.9002000000036787, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "ids": [], - "elapsedMs": 2.203449000000546, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - { - "id": "север-words", - "category": "words", - "ids": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867" - ], - "elapsedMs": 7.92291600000317, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - { - "id": "север-typo", - "category": "typo", - "ids": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867" - ], - "elapsedMs": 4.4926289999930304, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - { - "id": "север-beginnings", - "category": "beginnings", - "ids": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867" - ], - "elapsedMs": 2.7162700000044424, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - { - "id": "север-phrase", - "category": "phrase", - "ids": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862" - ], - "elapsedMs": 1.6422370000000228, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - { - "id": "север-from", - "category": "from", - "ids": [ - "10605" - ], - "elapsedMs": 3.113615000002028, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "север-chat", - "category": "chat", - "ids": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867" - ], - "elapsedMs": 3.446919000001799, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "север-date", - "category": "date", - "ids": [ - "10605", - "10606" - ], - "elapsedMs": 3.272292000001471, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "север-and", - "category": "and", - "ids": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867" - ], - "elapsedMs": 1.7656690000003437, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - { - "id": "север-or", - "category": "or", - "ids": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867" - ], - "elapsedMs": 1.7154039999950328, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "ids": [], - "elapsedMs": 2.115262999999686, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "ids": [], - "elapsedMs": 1.9881450000029872, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "ids": [], - "elapsedMs": 2.263823000001139, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - { - "id": "lumen-words", - "category": "words", - "ids": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159" - ], - "elapsedMs": 3.875832999998238, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-typo", - "category": "typo", - "ids": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159" - ], - "elapsedMs": 4.720746999999392, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "ids": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159" - ], - "elapsedMs": 2.649865999999747, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-phrase", - "category": "phrase", - "ids": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159" - ], - "elapsedMs": 1.5868439999976545, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-from", - "category": "from", - "ids": [ - "10907" - ], - "elapsedMs": 3.1440010000005714, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - { - "id": "lumen-chat", - "category": "chat", - "ids": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159" - ], - "elapsedMs": 3.2406519999931334, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-date", - "category": "date", - "ids": [ - "10908", - "10907" - ], - "elapsedMs": 3.1149070000028587, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - { - "id": "lumen-and", - "category": "and", - "ids": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159" - ], - "elapsedMs": 1.6945750000013504, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-or", - "category": "or", - "ids": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159" - ], - "elapsedMs": 1.7035219999961555, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "ids": [], - "elapsedMs": 2.048427999994601, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "ids": [], - "elapsedMs": 1.8857040000002598, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "ids": [], - "elapsedMs": 2.1373449999955483, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - ], - "heldOut": null -} diff --git a/bench/message-search-quality/combined.ts b/bench/message-search-quality/combined.ts deleted file mode 100644 index 3b97d0b67..000000000 --- a/bench/message-search-quality/combined.ts +++ /dev/null @@ -1,184 +0,0 @@ -import assert from "node:assert/strict" -import { createHash } from "node:crypto" -import { mkdtempSync, readFileSync } from "node:fs" -import { tmpdir } from "node:os" -import { join } from "node:path" -import { performance } from "node:perf_hooks" -import type { Message } from "../../dist/domain/models.js" -import { searchCombined, type CombinedOptions } from "../../dist/services/messages-combined.js" -import { searchStore } from "../../dist/services/messages.js" -import { openStore } from "../../dist/store/store.js" - -type Query = { id: string; split: "dev" | "test"; category: string; lucene: string; relevant: Record } -type Corpus = { - chats: { id: string; title: string }[] - messages: Pick[] - queries: Query[] -} -type Row = { - id: string - category: string - recallAt10: number | null - mrr: number | null - ndcgAt10: number | null - falseHits: number - ids: string[] - elapsedMs: number -} -const bytes = readFileSync(new URL("./corpus.json", import.meta.url)) -const corpus = JSON.parse(bytes.toString()) as Corpus -const root = mkdtempSync(join(tmpdir(), "combined-quality-")) -process.env.TZ = "UTC" -process.env.MESSAGING_STORE = join(root, "quality.db") -process.env.CLI_COMMON_CACHE_DIR = join(root, "empty-cache") -const store = await openStore({ path: process.env.MESSAGING_STORE }) -const account = { provider: "synthetic", account: "500" } -const report = (rows: Row[]) => { - const answerable = rows.filter((r) => r.recallAt10 !== null) - const average = (key: "recallAt10" | "mrr" | "ndcgAt10") => - answerable.reduce((s, r) => s + (r[key] ?? 0), 0) / answerable.length - const times = rows.map((r) => r.elapsedMs).sort((a, b) => a - b) - return { - recallAt10: average("recallAt10"), - mrr: average("mrr"), - ndcgAt10: average("ndcgAt10"), - noAnswerFalseHits: rows.reduce((s, r) => s + r.falseHits, 0), - p50Ms: times[Math.floor(times.length * 0.5)], - p95Ms: times[Math.floor(times.length * 0.95)], - } -} -const configurations: CombinedOptions[] = [] -for (const candidateDepth of [50, 100, 300, 500]) - for (const rrfK of [20, 60, 100]) - for (const rerank of ["none", "coverage", "proximity", "phrase", "all"] as const) - for (const chatCap of [0, 3, 5]) configurations.push({ candidateDepth, rrfK, rerank, chatCap }) -try { - await store.saveChats( - account, - corpus.chats.map((c) => ({ ...c, kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 100 })), - ) - for (const chat of corpus.chats) - await store.saveMessages( - account, - chat.id, - corpus.messages - .filter((m) => m.chatId === chat.id) - .map( - (m): Message => ({ - ...m, - senderName: `Synthetic ${m.senderId}`, - editedAt: null, - outgoing: false, - attachments: [], - replyTo: null, - forwardedFrom: null, - reactions: null, - }), - ), - { via: "synthetic" }, - ) - await store.fillSearchIndex({}) - await store.fillStems({}) - const run = async (options: CombinedOptions, queries: Query[]) => { - const rows: Row[] = [] - for (const q of queries) { - const start = performance.now() - const found = await searchCombined(store, account, { text: q.lucene, limit: 10, timezone: "UTC" }, {}, options) - const elapsedMs = performance.now() - start - const ids = found.items.map((h) => h.id) - assert.equal(new Set(ids).size, ids.length) - const grades = Object.values(q.relevant) - const first = ids.findIndex((id) => q.relevant[id]) - const dcg = (g: number[]) => g.reduce((s, v, i) => s + (2 ** v - 1) / Math.log2(i + 2), 0) - rows.push({ - id: q.id, - category: q.category, - ids, - elapsedMs, - recallAt10: grades.length ? ids.filter((id) => q.relevant[id]).length / grades.length : null, - mrr: grades.length ? (first < 0 ? 0 : 1 / (first + 1)) : null, - ndcgAt10: grades.length - ? dcg(ids.map((id) => q.relevant[id] ?? 0)) / dcg([...grades].sort((a, b) => b - a).slice(0, 10)) - : null, - falseHits: grades.length ? 0 : ids.length, - }) - if (["phrase", "and", "or"].includes(q.category)) { - const strict = await searchStore(store, account, { - text: q.lucene, - language: "lucene", - timezone: "UTC", - limit: 10, - }) - assert.deepEqual( - ids, - strict.items.map((h) => h.id), - `strict eligibility/order: ${q.id}`, - ) - } - } - return rows - } - const dev = corpus.queries.filter((q) => q.split === "dev") - const results = [] - for (const options of configurations) { - const rows = await run(options, dev) - results.push({ options, metrics: report(rows) }) - } - const meetsGates = (m: ReturnType) => - m.recallAt10 >= 0.45 && m.mrr >= 0.5 && m.ndcgAt10 >= 0.4 && m.noAnswerFalseHits === 0 - const eligible = results.filter((r) => meetsGates(r.metrics)) - const chosen = [...(eligible.length ? eligible : results)].sort((a, b) => - eligible.length - ? a.options.candidateDepth - b.options.candidateDepth || - b.metrics.ndcgAt10 - a.metrics.ndcgAt10 || - (a.metrics.p95Ms ?? 0) - (b.metrics.p95Ms ?? 0) - : b.metrics.ndcgAt10 - a.metrics.ndcgAt10 || - b.metrics.recallAt10 - a.metrics.recallAt10 || - a.options.candidateDepth - b.options.candidateDepth, - )[0] - assert.ok(chosen) - const chosenRows = await run(chosen.options, dev) - // No held-out evaluation until a dev configuration meets the approved release gates. - console.log( - JSON.stringify( - { - corpusSha256: createHash("sha256").update(bytes).digest("hex"), - productionSha256: Object.fromEntries( - [ - "services/messages-combined.js", - "services/messages-search.js", - "search/lucene/parser.js", - "search/correct.js", - "search/stem.js", - "store/normalize.js", - "store/sqlite/lucene.js", - ].map((file) => [ - file, - createHash("sha256") - .update(readFileSync(new URL(`../../dist/${file}`, import.meta.url))) - .digest("hex"), - ]), - ), - runnerSha256: createHash("sha256") - .update(readFileSync(new URL("./combined.ts", import.meta.url))) - .digest("hex"), - configurations: results, - devPassed: eligible.length > 0, - chosen, - devRankings: chosenRows, - heldOut: eligible.length - ? report( - await run( - chosen.options, - corpus.queries.filter((q) => q.split === "test"), - ), - ) - : null, - }, - null, - 2, - ), - ) -} finally { - await store.close() -} diff --git a/bench/message-search-quality/corpus.json b/bench/message-search-quality/corpus.json deleted file mode 100644 index fe361f1ab..000000000 --- a/bench/message-search-quality/corpus.json +++ /dev/null @@ -1,19452 +0,0 @@ -{ - "chats": [ - { - "id": "101", - "title": "Synthetic aurora decisions" - }, - { - "id": "102", - "title": "Synthetic cedar decisions" - }, - { - "id": "103", - "title": "Synthetic север decisions" - }, - { - "id": "104", - "title": "Synthetic lumen decisions" - }, - { - "id": "105", - "title": "Synthetic harbor decisions" - }, - { - "id": "106", - "title": "Synthetic maple decisions" - }, - { - "id": "107", - "title": "Synthetic вектор decisions" - }, - { - "id": "108", - "title": "Synthetic prism decisions" - }, - { - "id": "999", - "title": "Synthetic busy operations" - } - ], - "messages": [ - { - "id": "10001", - "chatId": "101", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Aurora rollout rollback decision: disable the canary and restore build 17." - }, - { - "id": "10002", - "chatId": "101", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "The Aurora rollout rollback owner is Mira; the decision is recorded above." - }, - { - "id": "10003", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 0." - }, - { - "id": "10004", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 1." - }, - { - "id": "10005", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 2." - }, - { - "id": "10006", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 3." - }, - { - "id": "10007", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "10008", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 5." - }, - { - "id": "10009", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 6." - }, - { - "id": "10010", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 7." - }, - { - "id": "10011", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 8." - }, - { - "id": "10012", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "10013", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 10." - }, - { - "id": "10014", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 11." - }, - { - "id": "10015", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 12." - }, - { - "id": "10016", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 13." - }, - { - "id": "10017", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "10018", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 15." - }, - { - "id": "10019", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 16." - }, - { - "id": "10020", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 17." - }, - { - "id": "10021", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 18." - }, - { - "id": "10022", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "10023", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 20." - }, - { - "id": "10024", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 21." - }, - { - "id": "10025", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 22." - }, - { - "id": "10026", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 23." - }, - { - "id": "10027", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "10028", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 25." - }, - { - "id": "10029", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 26." - }, - { - "id": "10030", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 27." - }, - { - "id": "10031", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 28." - }, - { - "id": "10032", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "10033", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 30." - }, - { - "id": "10034", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 31." - }, - { - "id": "10035", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 32." - }, - { - "id": "10036", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 33." - }, - { - "id": "10037", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "10038", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 35." - }, - { - "id": "10039", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 36." - }, - { - "id": "10040", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 37." - }, - { - "id": "10041", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 38." - }, - { - "id": "10042", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "10043", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 40." - }, - { - "id": "10044", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 41." - }, - { - "id": "10045", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 42." - }, - { - "id": "10046", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 43." - }, - { - "id": "10047", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "10048", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 45." - }, - { - "id": "10049", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 46." - }, - { - "id": "10050", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 47." - }, - { - "id": "10051", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 48." - }, - { - "id": "10052", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "10053", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 50." - }, - { - "id": "10054", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 51." - }, - { - "id": "10055", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 52." - }, - { - "id": "10056", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 53." - }, - { - "id": "10057", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "10058", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 55." - }, - { - "id": "10059", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 56." - }, - { - "id": "10060", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 57." - }, - { - "id": "10061", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 58." - }, - { - "id": "10062", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "10063", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 60." - }, - { - "id": "10064", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 61." - }, - { - "id": "10065", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 62." - }, - { - "id": "10066", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 63." - }, - { - "id": "10067", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "10068", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 65." - }, - { - "id": "10069", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 66." - }, - { - "id": "10070", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 67." - }, - { - "id": "10071", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 68." - }, - { - "id": "10072", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "10073", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 70." - }, - { - "id": "10074", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 71." - }, - { - "id": "10075", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 72." - }, - { - "id": "10076", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 73." - }, - { - "id": "10077", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "10078", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 75." - }, - { - "id": "10079", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 76." - }, - { - "id": "10080", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 77." - }, - { - "id": "10081", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 78." - }, - { - "id": "10082", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "10083", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 80." - }, - { - "id": "10084", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 81." - }, - { - "id": "10085", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 82." - }, - { - "id": "10086", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 83." - }, - { - "id": "10087", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "10088", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 85." - }, - { - "id": "10089", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 86." - }, - { - "id": "10090", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 87." - }, - { - "id": "10091", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 88." - }, - { - "id": "10092", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "10093", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 90." - }, - { - "id": "10094", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 91." - }, - { - "id": "10095", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 92." - }, - { - "id": "10096", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 93." - }, - { - "id": "10097", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "10098", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 95." - }, - { - "id": "10099", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 96." - }, - { - "id": "10100", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 97." - }, - { - "id": "10101", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 98." - }, - { - "id": "10102", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "10103", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 100." - }, - { - "id": "10104", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 101." - }, - { - "id": "10105", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 102." - }, - { - "id": "10106", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 103." - }, - { - "id": "10107", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "10108", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 105." - }, - { - "id": "10109", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 106." - }, - { - "id": "10110", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 107." - }, - { - "id": "10111", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 108." - }, - { - "id": "10112", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "10113", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 110." - }, - { - "id": "10114", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 111." - }, - { - "id": "10115", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 112." - }, - { - "id": "10116", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 113." - }, - { - "id": "10117", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "10118", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 115." - }, - { - "id": "10119", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 116." - }, - { - "id": "10120", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 117." - }, - { - "id": "10121", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 118." - }, - { - "id": "10122", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "10123", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 120." - }, - { - "id": "10124", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 121." - }, - { - "id": "10125", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 122." - }, - { - "id": "10126", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 123." - }, - { - "id": "10127", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "10128", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 125." - }, - { - "id": "10129", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 126." - }, - { - "id": "10130", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 127." - }, - { - "id": "10131", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 128." - }, - { - "id": "10132", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "10133", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 130." - }, - { - "id": "10134", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 131." - }, - { - "id": "10135", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 132." - }, - { - "id": "10136", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 133." - }, - { - "id": "10137", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "10138", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 135." - }, - { - "id": "10139", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 136." - }, - { - "id": "10140", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 137." - }, - { - "id": "10141", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 138." - }, - { - "id": "10142", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "10143", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 140." - }, - { - "id": "10144", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 141." - }, - { - "id": "10145", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 142." - }, - { - "id": "10146", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 143." - }, - { - "id": "10147", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "10148", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 145." - }, - { - "id": "10149", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 146." - }, - { - "id": "10150", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 147." - }, - { - "id": "10151", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 148." - }, - { - "id": "10152", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "10153", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 150." - }, - { - "id": "10154", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 151." - }, - { - "id": "10155", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 152." - }, - { - "id": "10156", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 153." - }, - { - "id": "10157", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "10158", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 155." - }, - { - "id": "10159", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 156." - }, - { - "id": "10160", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 157." - }, - { - "id": "10161", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 158." - }, - { - "id": "10162", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "10163", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 160." - }, - { - "id": "10164", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 161." - }, - { - "id": "10165", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 162." - }, - { - "id": "10166", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 163." - }, - { - "id": "10167", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "10168", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 165." - }, - { - "id": "10169", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 166." - }, - { - "id": "10170", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 167." - }, - { - "id": "10171", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 168." - }, - { - "id": "10172", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "10173", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 170." - }, - { - "id": "10174", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 171." - }, - { - "id": "10175", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 172." - }, - { - "id": "10176", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 173." - }, - { - "id": "10177", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "10178", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 175." - }, - { - "id": "10179", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 176." - }, - { - "id": "10180", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 177." - }, - { - "id": "10181", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 178." - }, - { - "id": "10182", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "10183", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 180." - }, - { - "id": "10184", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 181." - }, - { - "id": "10185", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 182." - }, - { - "id": "10186", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 183." - }, - { - "id": "10187", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "10188", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 185." - }, - { - "id": "10189", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 186." - }, - { - "id": "10190", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 187." - }, - { - "id": "10191", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 188." - }, - { - "id": "10192", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "10193", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 190." - }, - { - "id": "10194", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 191." - }, - { - "id": "10195", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 192." - }, - { - "id": "10196", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 193." - }, - { - "id": "10197", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "10198", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 195." - }, - { - "id": "10199", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 196." - }, - { - "id": "10200", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 197." - }, - { - "id": "10201", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 198." - }, - { - "id": "10202", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "10203", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 200." - }, - { - "id": "10204", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 201." - }, - { - "id": "10205", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 202." - }, - { - "id": "10206", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 203." - }, - { - "id": "10207", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "10208", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 205." - }, - { - "id": "10209", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 206." - }, - { - "id": "10210", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 207." - }, - { - "id": "10211", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 208." - }, - { - "id": "10212", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "10213", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 210." - }, - { - "id": "10214", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 211." - }, - { - "id": "10215", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 212." - }, - { - "id": "10216", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 213." - }, - { - "id": "10217", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "10218", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 215." - }, - { - "id": "10219", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 216." - }, - { - "id": "10220", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 217." - }, - { - "id": "10221", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 218." - }, - { - "id": "10222", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "10223", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 220." - }, - { - "id": "10224", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 221." - }, - { - "id": "10225", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 222." - }, - { - "id": "10226", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 223." - }, - { - "id": "10227", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "10228", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 225." - }, - { - "id": "10229", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 226." - }, - { - "id": "10230", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 227." - }, - { - "id": "10231", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 228." - }, - { - "id": "10232", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "10233", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 230." - }, - { - "id": "10234", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 231." - }, - { - "id": "10235", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 232." - }, - { - "id": "10236", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 233." - }, - { - "id": "10237", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "10238", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 235." - }, - { - "id": "10239", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 236." - }, - { - "id": "10240", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 237." - }, - { - "id": "10241", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 238." - }, - { - "id": "10242", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "10243", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 240." - }, - { - "id": "10244", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 241." - }, - { - "id": "10245", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 242." - }, - { - "id": "10246", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 243." - }, - { - "id": "10247", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "10248", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 245." - }, - { - "id": "10249", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 246." - }, - { - "id": "10250", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 247." - }, - { - "id": "10251", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 248." - }, - { - "id": "10252", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "10253", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 250." - }, - { - "id": "10254", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 251." - }, - { - "id": "10255", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 252." - }, - { - "id": "10256", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 253." - }, - { - "id": "10257", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "10258", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 255." - }, - { - "id": "10259", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 256." - }, - { - "id": "10260", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 257." - }, - { - "id": "10261", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 258." - }, - { - "id": "10262", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "10263", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 260." - }, - { - "id": "10264", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 261." - }, - { - "id": "10265", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 262." - }, - { - "id": "10266", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 263." - }, - { - "id": "10267", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "10268", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 265." - }, - { - "id": "10269", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 266." - }, - { - "id": "10270", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 267." - }, - { - "id": "10271", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 268." - }, - { - "id": "10272", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "10273", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 270." - }, - { - "id": "10274", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 271." - }, - { - "id": "10275", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 272." - }, - { - "id": "10276", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 273." - }, - { - "id": "10277", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "10278", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 275." - }, - { - "id": "10279", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 276." - }, - { - "id": "10280", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 277." - }, - { - "id": "10281", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 278." - }, - { - "id": "10282", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "10283", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 280." - }, - { - "id": "10284", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 281." - }, - { - "id": "10285", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 282." - }, - { - "id": "10286", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 283." - }, - { - "id": "10287", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "10288", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 285." - }, - { - "id": "10289", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 286." - }, - { - "id": "10290", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 287." - }, - { - "id": "10291", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 288." - }, - { - "id": "10292", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "10293", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 290." - }, - { - "id": "10294", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 291." - }, - { - "id": "10295", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 292." - }, - { - "id": "10296", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 293." - }, - { - "id": "10297", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "10298", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback: agenda only; no decision yet. Thread 295." - }, - { - "id": "10299", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora rollout rollback? Please add this question to the meeting. Thread 296." - }, - { - "id": "10300", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: aurora rollout rollback. This empty template supplies no policy. Thread 297." - }, - { - "id": "10301", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "aurora status digest. rollout is a document tag. rollback is a separate training exercise. No answer. Thread 298." - }, - { - "id": "10302", - "chatId": "101", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old aurora rollout rollback proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "10303", - "chatId": "102", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Cedar backup retention decision: keep daily snapshots for 35 days." - }, - { - "id": "10304", - "chatId": "102", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "The Cedar backup retention owner is Niko; the policy is recorded above." - }, - { - "id": "10305", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 0." - }, - { - "id": "10306", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 1." - }, - { - "id": "10307", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 2." - }, - { - "id": "10308", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 3." - }, - { - "id": "10309", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "10310", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 5." - }, - { - "id": "10311", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 6." - }, - { - "id": "10312", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 7." - }, - { - "id": "10313", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 8." - }, - { - "id": "10314", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "10315", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 10." - }, - { - "id": "10316", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 11." - }, - { - "id": "10317", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 12." - }, - { - "id": "10318", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 13." - }, - { - "id": "10319", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "10320", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 15." - }, - { - "id": "10321", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 16." - }, - { - "id": "10322", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 17." - }, - { - "id": "10323", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 18." - }, - { - "id": "10324", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "10325", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 20." - }, - { - "id": "10326", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 21." - }, - { - "id": "10327", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 22." - }, - { - "id": "10328", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 23." - }, - { - "id": "10329", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "10330", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 25." - }, - { - "id": "10331", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 26." - }, - { - "id": "10332", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 27." - }, - { - "id": "10333", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 28." - }, - { - "id": "10334", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "10335", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 30." - }, - { - "id": "10336", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 31." - }, - { - "id": "10337", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 32." - }, - { - "id": "10338", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 33." - }, - { - "id": "10339", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "10340", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 35." - }, - { - "id": "10341", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 36." - }, - { - "id": "10342", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 37." - }, - { - "id": "10343", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 38." - }, - { - "id": "10344", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "10345", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 40." - }, - { - "id": "10346", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 41." - }, - { - "id": "10347", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 42." - }, - { - "id": "10348", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 43." - }, - { - "id": "10349", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "10350", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 45." - }, - { - "id": "10351", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 46." - }, - { - "id": "10352", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 47." - }, - { - "id": "10353", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 48." - }, - { - "id": "10354", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "10355", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 50." - }, - { - "id": "10356", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 51." - }, - { - "id": "10357", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 52." - }, - { - "id": "10358", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 53." - }, - { - "id": "10359", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "10360", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 55." - }, - { - "id": "10361", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 56." - }, - { - "id": "10362", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 57." - }, - { - "id": "10363", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 58." - }, - { - "id": "10364", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "10365", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 60." - }, - { - "id": "10366", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 61." - }, - { - "id": "10367", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 62." - }, - { - "id": "10368", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 63." - }, - { - "id": "10369", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "10370", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 65." - }, - { - "id": "10371", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 66." - }, - { - "id": "10372", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 67." - }, - { - "id": "10373", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 68." - }, - { - "id": "10374", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "10375", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 70." - }, - { - "id": "10376", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 71." - }, - { - "id": "10377", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 72." - }, - { - "id": "10378", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 73." - }, - { - "id": "10379", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "10380", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 75." - }, - { - "id": "10381", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 76." - }, - { - "id": "10382", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 77." - }, - { - "id": "10383", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 78." - }, - { - "id": "10384", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "10385", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 80." - }, - { - "id": "10386", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 81." - }, - { - "id": "10387", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 82." - }, - { - "id": "10388", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 83." - }, - { - "id": "10389", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "10390", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 85." - }, - { - "id": "10391", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 86." - }, - { - "id": "10392", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 87." - }, - { - "id": "10393", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 88." - }, - { - "id": "10394", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "10395", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 90." - }, - { - "id": "10396", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 91." - }, - { - "id": "10397", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 92." - }, - { - "id": "10398", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 93." - }, - { - "id": "10399", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "10400", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 95." - }, - { - "id": "10401", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 96." - }, - { - "id": "10402", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 97." - }, - { - "id": "10403", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 98." - }, - { - "id": "10404", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "10405", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 100." - }, - { - "id": "10406", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 101." - }, - { - "id": "10407", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 102." - }, - { - "id": "10408", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 103." - }, - { - "id": "10409", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "10410", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 105." - }, - { - "id": "10411", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 106." - }, - { - "id": "10412", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 107." - }, - { - "id": "10413", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 108." - }, - { - "id": "10414", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "10415", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 110." - }, - { - "id": "10416", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 111." - }, - { - "id": "10417", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 112." - }, - { - "id": "10418", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 113." - }, - { - "id": "10419", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "10420", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 115." - }, - { - "id": "10421", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 116." - }, - { - "id": "10422", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 117." - }, - { - "id": "10423", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 118." - }, - { - "id": "10424", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "10425", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 120." - }, - { - "id": "10426", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 121." - }, - { - "id": "10427", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 122." - }, - { - "id": "10428", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 123." - }, - { - "id": "10429", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "10430", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 125." - }, - { - "id": "10431", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 126." - }, - { - "id": "10432", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 127." - }, - { - "id": "10433", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 128." - }, - { - "id": "10434", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "10435", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 130." - }, - { - "id": "10436", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 131." - }, - { - "id": "10437", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 132." - }, - { - "id": "10438", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 133." - }, - { - "id": "10439", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "10440", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 135." - }, - { - "id": "10441", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 136." - }, - { - "id": "10442", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 137." - }, - { - "id": "10443", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 138." - }, - { - "id": "10444", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "10445", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 140." - }, - { - "id": "10446", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 141." - }, - { - "id": "10447", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 142." - }, - { - "id": "10448", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 143." - }, - { - "id": "10449", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "10450", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 145." - }, - { - "id": "10451", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 146." - }, - { - "id": "10452", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 147." - }, - { - "id": "10453", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 148." - }, - { - "id": "10454", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "10455", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 150." - }, - { - "id": "10456", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 151." - }, - { - "id": "10457", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 152." - }, - { - "id": "10458", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 153." - }, - { - "id": "10459", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "10460", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 155." - }, - { - "id": "10461", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 156." - }, - { - "id": "10462", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 157." - }, - { - "id": "10463", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 158." - }, - { - "id": "10464", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "10465", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 160." - }, - { - "id": "10466", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 161." - }, - { - "id": "10467", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 162." - }, - { - "id": "10468", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 163." - }, - { - "id": "10469", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "10470", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 165." - }, - { - "id": "10471", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 166." - }, - { - "id": "10472", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 167." - }, - { - "id": "10473", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 168." - }, - { - "id": "10474", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "10475", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 170." - }, - { - "id": "10476", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 171." - }, - { - "id": "10477", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 172." - }, - { - "id": "10478", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 173." - }, - { - "id": "10479", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "10480", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 175." - }, - { - "id": "10481", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 176." - }, - { - "id": "10482", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 177." - }, - { - "id": "10483", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 178." - }, - { - "id": "10484", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "10485", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 180." - }, - { - "id": "10486", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 181." - }, - { - "id": "10487", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 182." - }, - { - "id": "10488", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 183." - }, - { - "id": "10489", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "10490", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 185." - }, - { - "id": "10491", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 186." - }, - { - "id": "10492", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 187." - }, - { - "id": "10493", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 188." - }, - { - "id": "10494", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "10495", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 190." - }, - { - "id": "10496", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 191." - }, - { - "id": "10497", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 192." - }, - { - "id": "10498", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 193." - }, - { - "id": "10499", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "10500", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 195." - }, - { - "id": "10501", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 196." - }, - { - "id": "10502", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 197." - }, - { - "id": "10503", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 198." - }, - { - "id": "10504", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "10505", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 200." - }, - { - "id": "10506", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 201." - }, - { - "id": "10507", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 202." - }, - { - "id": "10508", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 203." - }, - { - "id": "10509", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "10510", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 205." - }, - { - "id": "10511", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 206." - }, - { - "id": "10512", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 207." - }, - { - "id": "10513", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 208." - }, - { - "id": "10514", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "10515", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 210." - }, - { - "id": "10516", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 211." - }, - { - "id": "10517", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 212." - }, - { - "id": "10518", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 213." - }, - { - "id": "10519", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "10520", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 215." - }, - { - "id": "10521", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 216." - }, - { - "id": "10522", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 217." - }, - { - "id": "10523", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 218." - }, - { - "id": "10524", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "10525", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 220." - }, - { - "id": "10526", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 221." - }, - { - "id": "10527", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 222." - }, - { - "id": "10528", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 223." - }, - { - "id": "10529", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "10530", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 225." - }, - { - "id": "10531", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 226." - }, - { - "id": "10532", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 227." - }, - { - "id": "10533", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 228." - }, - { - "id": "10534", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "10535", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 230." - }, - { - "id": "10536", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 231." - }, - { - "id": "10537", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 232." - }, - { - "id": "10538", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 233." - }, - { - "id": "10539", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "10540", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 235." - }, - { - "id": "10541", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 236." - }, - { - "id": "10542", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 237." - }, - { - "id": "10543", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 238." - }, - { - "id": "10544", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "10545", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 240." - }, - { - "id": "10546", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 241." - }, - { - "id": "10547", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 242." - }, - { - "id": "10548", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 243." - }, - { - "id": "10549", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "10550", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 245." - }, - { - "id": "10551", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 246." - }, - { - "id": "10552", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 247." - }, - { - "id": "10553", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 248." - }, - { - "id": "10554", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "10555", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 250." - }, - { - "id": "10556", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 251." - }, - { - "id": "10557", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 252." - }, - { - "id": "10558", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 253." - }, - { - "id": "10559", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "10560", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 255." - }, - { - "id": "10561", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 256." - }, - { - "id": "10562", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 257." - }, - { - "id": "10563", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 258." - }, - { - "id": "10564", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "10565", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 260." - }, - { - "id": "10566", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 261." - }, - { - "id": "10567", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 262." - }, - { - "id": "10568", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 263." - }, - { - "id": "10569", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "10570", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 265." - }, - { - "id": "10571", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 266." - }, - { - "id": "10572", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 267." - }, - { - "id": "10573", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 268." - }, - { - "id": "10574", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "10575", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 270." - }, - { - "id": "10576", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 271." - }, - { - "id": "10577", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 272." - }, - { - "id": "10578", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 273." - }, - { - "id": "10579", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "10580", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 275." - }, - { - "id": "10581", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 276." - }, - { - "id": "10582", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 277." - }, - { - "id": "10583", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 278." - }, - { - "id": "10584", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "10585", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 280." - }, - { - "id": "10586", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 281." - }, - { - "id": "10587", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 282." - }, - { - "id": "10588", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 283." - }, - { - "id": "10589", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "10590", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 285." - }, - { - "id": "10591", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 286." - }, - { - "id": "10592", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 287." - }, - { - "id": "10593", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 288." - }, - { - "id": "10594", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "10595", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 290." - }, - { - "id": "10596", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 291." - }, - { - "id": "10597", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 292." - }, - { - "id": "10598", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 293." - }, - { - "id": "10599", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "10600", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention: agenda only; no decision yet. Thread 295." - }, - { - "id": "10601", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar backup retention? Please add this question to the meeting. Thread 296." - }, - { - "id": "10602", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: cedar backup retention. This empty template supplies no policy. Thread 297." - }, - { - "id": "10603", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "cedar status digest. backup is a document tag. retention is a separate training exercise. No answer. Thread 298." - }, - { - "id": "10604", - "chatId": "102", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old cedar backup retention proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "10605", - "chatId": "103", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря." - }, - { - "id": "10606", - "chatId": "103", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Север сертификат обновление проверено, владелец Ира." - }, - { - "id": "10607", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 0." - }, - { - "id": "10608", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 1." - }, - { - "id": "10609", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 2." - }, - { - "id": "10610", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 3." - }, - { - "id": "10611", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "10612", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 5." - }, - { - "id": "10613", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 6." - }, - { - "id": "10614", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 7." - }, - { - "id": "10615", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 8." - }, - { - "id": "10616", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "10617", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 10." - }, - { - "id": "10618", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 11." - }, - { - "id": "10619", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 12." - }, - { - "id": "10620", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 13." - }, - { - "id": "10621", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "10622", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 15." - }, - { - "id": "10623", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 16." - }, - { - "id": "10624", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 17." - }, - { - "id": "10625", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 18." - }, - { - "id": "10626", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "10627", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 20." - }, - { - "id": "10628", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 21." - }, - { - "id": "10629", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 22." - }, - { - "id": "10630", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 23." - }, - { - "id": "10631", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "10632", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 25." - }, - { - "id": "10633", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 26." - }, - { - "id": "10634", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 27." - }, - { - "id": "10635", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 28." - }, - { - "id": "10636", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "10637", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 30." - }, - { - "id": "10638", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 31." - }, - { - "id": "10639", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 32." - }, - { - "id": "10640", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 33." - }, - { - "id": "10641", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "10642", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 35." - }, - { - "id": "10643", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 36." - }, - { - "id": "10644", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 37." - }, - { - "id": "10645", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 38." - }, - { - "id": "10646", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "10647", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 40." - }, - { - "id": "10648", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 41." - }, - { - "id": "10649", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 42." - }, - { - "id": "10650", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 43." - }, - { - "id": "10651", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "10652", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 45." - }, - { - "id": "10653", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 46." - }, - { - "id": "10654", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 47." - }, - { - "id": "10655", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 48." - }, - { - "id": "10656", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "10657", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 50." - }, - { - "id": "10658", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 51." - }, - { - "id": "10659", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 52." - }, - { - "id": "10660", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 53." - }, - { - "id": "10661", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "10662", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 55." - }, - { - "id": "10663", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 56." - }, - { - "id": "10664", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 57." - }, - { - "id": "10665", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 58." - }, - { - "id": "10666", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "10667", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 60." - }, - { - "id": "10668", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 61." - }, - { - "id": "10669", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 62." - }, - { - "id": "10670", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 63." - }, - { - "id": "10671", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "10672", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 65." - }, - { - "id": "10673", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 66." - }, - { - "id": "10674", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 67." - }, - { - "id": "10675", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 68." - }, - { - "id": "10676", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "10677", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 70." - }, - { - "id": "10678", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 71." - }, - { - "id": "10679", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 72." - }, - { - "id": "10680", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 73." - }, - { - "id": "10681", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "10682", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 75." - }, - { - "id": "10683", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 76." - }, - { - "id": "10684", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 77." - }, - { - "id": "10685", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 78." - }, - { - "id": "10686", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "10687", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 80." - }, - { - "id": "10688", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 81." - }, - { - "id": "10689", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 82." - }, - { - "id": "10690", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 83." - }, - { - "id": "10691", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "10692", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 85." - }, - { - "id": "10693", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 86." - }, - { - "id": "10694", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 87." - }, - { - "id": "10695", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 88." - }, - { - "id": "10696", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "10697", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 90." - }, - { - "id": "10698", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 91." - }, - { - "id": "10699", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 92." - }, - { - "id": "10700", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 93." - }, - { - "id": "10701", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "10702", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 95." - }, - { - "id": "10703", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 96." - }, - { - "id": "10704", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 97." - }, - { - "id": "10705", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 98." - }, - { - "id": "10706", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "10707", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 100." - }, - { - "id": "10708", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 101." - }, - { - "id": "10709", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 102." - }, - { - "id": "10710", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 103." - }, - { - "id": "10711", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "10712", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 105." - }, - { - "id": "10713", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 106." - }, - { - "id": "10714", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 107." - }, - { - "id": "10715", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 108." - }, - { - "id": "10716", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "10717", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 110." - }, - { - "id": "10718", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 111." - }, - { - "id": "10719", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 112." - }, - { - "id": "10720", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 113." - }, - { - "id": "10721", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "10722", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 115." - }, - { - "id": "10723", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 116." - }, - { - "id": "10724", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 117." - }, - { - "id": "10725", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 118." - }, - { - "id": "10726", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "10727", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 120." - }, - { - "id": "10728", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 121." - }, - { - "id": "10729", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 122." - }, - { - "id": "10730", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 123." - }, - { - "id": "10731", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "10732", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 125." - }, - { - "id": "10733", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 126." - }, - { - "id": "10734", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 127." - }, - { - "id": "10735", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 128." - }, - { - "id": "10736", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "10737", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 130." - }, - { - "id": "10738", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 131." - }, - { - "id": "10739", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 132." - }, - { - "id": "10740", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 133." - }, - { - "id": "10741", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "10742", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 135." - }, - { - "id": "10743", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 136." - }, - { - "id": "10744", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 137." - }, - { - "id": "10745", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 138." - }, - { - "id": "10746", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "10747", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 140." - }, - { - "id": "10748", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 141." - }, - { - "id": "10749", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 142." - }, - { - "id": "10750", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 143." - }, - { - "id": "10751", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "10752", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 145." - }, - { - "id": "10753", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 146." - }, - { - "id": "10754", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 147." - }, - { - "id": "10755", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 148." - }, - { - "id": "10756", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "10757", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 150." - }, - { - "id": "10758", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 151." - }, - { - "id": "10759", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 152." - }, - { - "id": "10760", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 153." - }, - { - "id": "10761", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "10762", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 155." - }, - { - "id": "10763", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 156." - }, - { - "id": "10764", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 157." - }, - { - "id": "10765", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 158." - }, - { - "id": "10766", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "10767", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 160." - }, - { - "id": "10768", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 161." - }, - { - "id": "10769", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 162." - }, - { - "id": "10770", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 163." - }, - { - "id": "10771", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "10772", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 165." - }, - { - "id": "10773", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 166." - }, - { - "id": "10774", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 167." - }, - { - "id": "10775", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 168." - }, - { - "id": "10776", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "10777", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 170." - }, - { - "id": "10778", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 171." - }, - { - "id": "10779", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 172." - }, - { - "id": "10780", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 173." - }, - { - "id": "10781", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "10782", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 175." - }, - { - "id": "10783", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 176." - }, - { - "id": "10784", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 177." - }, - { - "id": "10785", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 178." - }, - { - "id": "10786", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "10787", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 180." - }, - { - "id": "10788", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 181." - }, - { - "id": "10789", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 182." - }, - { - "id": "10790", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 183." - }, - { - "id": "10791", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "10792", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 185." - }, - { - "id": "10793", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 186." - }, - { - "id": "10794", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 187." - }, - { - "id": "10795", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 188." - }, - { - "id": "10796", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "10797", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 190." - }, - { - "id": "10798", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 191." - }, - { - "id": "10799", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 192." - }, - { - "id": "10800", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 193." - }, - { - "id": "10801", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "10802", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 195." - }, - { - "id": "10803", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 196." - }, - { - "id": "10804", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 197." - }, - { - "id": "10805", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 198." - }, - { - "id": "10806", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "10807", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 200." - }, - { - "id": "10808", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 201." - }, - { - "id": "10809", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 202." - }, - { - "id": "10810", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 203." - }, - { - "id": "10811", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "10812", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 205." - }, - { - "id": "10813", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 206." - }, - { - "id": "10814", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 207." - }, - { - "id": "10815", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 208." - }, - { - "id": "10816", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "10817", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 210." - }, - { - "id": "10818", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 211." - }, - { - "id": "10819", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 212." - }, - { - "id": "10820", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 213." - }, - { - "id": "10821", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "10822", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 215." - }, - { - "id": "10823", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 216." - }, - { - "id": "10824", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 217." - }, - { - "id": "10825", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 218." - }, - { - "id": "10826", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "10827", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 220." - }, - { - "id": "10828", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 221." - }, - { - "id": "10829", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 222." - }, - { - "id": "10830", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 223." - }, - { - "id": "10831", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "10832", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 225." - }, - { - "id": "10833", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 226." - }, - { - "id": "10834", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 227." - }, - { - "id": "10835", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 228." - }, - { - "id": "10836", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "10837", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 230." - }, - { - "id": "10838", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 231." - }, - { - "id": "10839", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 232." - }, - { - "id": "10840", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 233." - }, - { - "id": "10841", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "10842", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 235." - }, - { - "id": "10843", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 236." - }, - { - "id": "10844", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 237." - }, - { - "id": "10845", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 238." - }, - { - "id": "10846", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "10847", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 240." - }, - { - "id": "10848", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 241." - }, - { - "id": "10849", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 242." - }, - { - "id": "10850", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 243." - }, - { - "id": "10851", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "10852", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 245." - }, - { - "id": "10853", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 246." - }, - { - "id": "10854", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 247." - }, - { - "id": "10855", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 248." - }, - { - "id": "10856", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "10857", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 250." - }, - { - "id": "10858", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 251." - }, - { - "id": "10859", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 252." - }, - { - "id": "10860", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 253." - }, - { - "id": "10861", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "10862", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 255." - }, - { - "id": "10863", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 256." - }, - { - "id": "10864", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 257." - }, - { - "id": "10865", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 258." - }, - { - "id": "10866", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "10867", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 260." - }, - { - "id": "10868", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 261." - }, - { - "id": "10869", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 262." - }, - { - "id": "10870", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 263." - }, - { - "id": "10871", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "10872", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 265." - }, - { - "id": "10873", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 266." - }, - { - "id": "10874", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 267." - }, - { - "id": "10875", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 268." - }, - { - "id": "10876", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "10877", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 270." - }, - { - "id": "10878", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 271." - }, - { - "id": "10879", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 272." - }, - { - "id": "10880", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 273." - }, - { - "id": "10881", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "10882", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 275." - }, - { - "id": "10883", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 276." - }, - { - "id": "10884", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 277." - }, - { - "id": "10885", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 278." - }, - { - "id": "10886", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "10887", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 280." - }, - { - "id": "10888", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 281." - }, - { - "id": "10889", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 282." - }, - { - "id": "10890", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 283." - }, - { - "id": "10891", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "10892", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 285." - }, - { - "id": "10893", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 286." - }, - { - "id": "10894", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 287." - }, - { - "id": "10895", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 288." - }, - { - "id": "10896", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "10897", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 290." - }, - { - "id": "10898", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 291." - }, - { - "id": "10899", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 292." - }, - { - "id": "10900", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 293." - }, - { - "id": "10901", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "10902", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление: agenda only; no decision yet. Thread 295." - }, - { - "id": "10903", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север сертификат обновление? Please add this question to the meeting. Thread 296." - }, - { - "id": "10904", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: север сертификат обновление. This empty template supplies no policy. Thread 297." - }, - { - "id": "10905", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "север status digest. сертификат is a document tag. обновление is a separate training exercise. No answer. Thread 298." - }, - { - "id": "10906", - "chatId": "103", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old север сертификат обновление proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "10907", - "chatId": "104", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Lumen invoice refund approved: return 480 credits to the original payer." - }, - { - "id": "10908", - "chatId": "104", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "The Lumen invoice refund owner is Eva; approval is recorded above." - }, - { - "id": "10909", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 0." - }, - { - "id": "10910", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 1." - }, - { - "id": "10911", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 2." - }, - { - "id": "10912", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 3." - }, - { - "id": "10913", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "10914", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 5." - }, - { - "id": "10915", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 6." - }, - { - "id": "10916", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 7." - }, - { - "id": "10917", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 8." - }, - { - "id": "10918", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "10919", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 10." - }, - { - "id": "10920", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 11." - }, - { - "id": "10921", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 12." - }, - { - "id": "10922", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 13." - }, - { - "id": "10923", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "10924", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 15." - }, - { - "id": "10925", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 16." - }, - { - "id": "10926", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 17." - }, - { - "id": "10927", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 18." - }, - { - "id": "10928", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "10929", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 20." - }, - { - "id": "10930", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 21." - }, - { - "id": "10931", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 22." - }, - { - "id": "10932", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 23." - }, - { - "id": "10933", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "10934", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 25." - }, - { - "id": "10935", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 26." - }, - { - "id": "10936", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 27." - }, - { - "id": "10937", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 28." - }, - { - "id": "10938", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "10939", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 30." - }, - { - "id": "10940", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 31." - }, - { - "id": "10941", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 32." - }, - { - "id": "10942", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 33." - }, - { - "id": "10943", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "10944", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 35." - }, - { - "id": "10945", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 36." - }, - { - "id": "10946", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 37." - }, - { - "id": "10947", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 38." - }, - { - "id": "10948", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "10949", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 40." - }, - { - "id": "10950", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 41." - }, - { - "id": "10951", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 42." - }, - { - "id": "10952", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 43." - }, - { - "id": "10953", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "10954", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 45." - }, - { - "id": "10955", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 46." - }, - { - "id": "10956", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 47." - }, - { - "id": "10957", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 48." - }, - { - "id": "10958", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "10959", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 50." - }, - { - "id": "10960", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 51." - }, - { - "id": "10961", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 52." - }, - { - "id": "10962", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 53." - }, - { - "id": "10963", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "10964", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 55." - }, - { - "id": "10965", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 56." - }, - { - "id": "10966", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 57." - }, - { - "id": "10967", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 58." - }, - { - "id": "10968", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "10969", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 60." - }, - { - "id": "10970", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 61." - }, - { - "id": "10971", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 62." - }, - { - "id": "10972", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 63." - }, - { - "id": "10973", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "10974", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 65." - }, - { - "id": "10975", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 66." - }, - { - "id": "10976", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 67." - }, - { - "id": "10977", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 68." - }, - { - "id": "10978", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "10979", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 70." - }, - { - "id": "10980", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 71." - }, - { - "id": "10981", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 72." - }, - { - "id": "10982", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 73." - }, - { - "id": "10983", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "10984", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 75." - }, - { - "id": "10985", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 76." - }, - { - "id": "10986", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 77." - }, - { - "id": "10987", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 78." - }, - { - "id": "10988", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "10989", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 80." - }, - { - "id": "10990", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 81." - }, - { - "id": "10991", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 82." - }, - { - "id": "10992", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 83." - }, - { - "id": "10993", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "10994", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 85." - }, - { - "id": "10995", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 86." - }, - { - "id": "10996", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 87." - }, - { - "id": "10997", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 88." - }, - { - "id": "10998", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "10999", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 90." - }, - { - "id": "11000", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 91." - }, - { - "id": "11001", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 92." - }, - { - "id": "11002", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 93." - }, - { - "id": "11003", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "11004", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 95." - }, - { - "id": "11005", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 96." - }, - { - "id": "11006", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 97." - }, - { - "id": "11007", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 98." - }, - { - "id": "11008", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "11009", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 100." - }, - { - "id": "11010", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 101." - }, - { - "id": "11011", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 102." - }, - { - "id": "11012", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 103." - }, - { - "id": "11013", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "11014", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 105." - }, - { - "id": "11015", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 106." - }, - { - "id": "11016", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 107." - }, - { - "id": "11017", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 108." - }, - { - "id": "11018", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "11019", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 110." - }, - { - "id": "11020", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 111." - }, - { - "id": "11021", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 112." - }, - { - "id": "11022", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 113." - }, - { - "id": "11023", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "11024", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 115." - }, - { - "id": "11025", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 116." - }, - { - "id": "11026", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 117." - }, - { - "id": "11027", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 118." - }, - { - "id": "11028", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "11029", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 120." - }, - { - "id": "11030", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 121." - }, - { - "id": "11031", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 122." - }, - { - "id": "11032", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 123." - }, - { - "id": "11033", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "11034", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 125." - }, - { - "id": "11035", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 126." - }, - { - "id": "11036", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 127." - }, - { - "id": "11037", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 128." - }, - { - "id": "11038", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "11039", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 130." - }, - { - "id": "11040", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 131." - }, - { - "id": "11041", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 132." - }, - { - "id": "11042", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 133." - }, - { - "id": "11043", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "11044", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 135." - }, - { - "id": "11045", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 136." - }, - { - "id": "11046", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 137." - }, - { - "id": "11047", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 138." - }, - { - "id": "11048", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "11049", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 140." - }, - { - "id": "11050", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 141." - }, - { - "id": "11051", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 142." - }, - { - "id": "11052", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 143." - }, - { - "id": "11053", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "11054", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 145." - }, - { - "id": "11055", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 146." - }, - { - "id": "11056", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 147." - }, - { - "id": "11057", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 148." - }, - { - "id": "11058", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "11059", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 150." - }, - { - "id": "11060", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 151." - }, - { - "id": "11061", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 152." - }, - { - "id": "11062", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 153." - }, - { - "id": "11063", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "11064", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 155." - }, - { - "id": "11065", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 156." - }, - { - "id": "11066", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 157." - }, - { - "id": "11067", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 158." - }, - { - "id": "11068", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "11069", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 160." - }, - { - "id": "11070", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 161." - }, - { - "id": "11071", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 162." - }, - { - "id": "11072", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 163." - }, - { - "id": "11073", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "11074", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 165." - }, - { - "id": "11075", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 166." - }, - { - "id": "11076", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 167." - }, - { - "id": "11077", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 168." - }, - { - "id": "11078", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "11079", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 170." - }, - { - "id": "11080", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 171." - }, - { - "id": "11081", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 172." - }, - { - "id": "11082", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 173." - }, - { - "id": "11083", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "11084", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 175." - }, - { - "id": "11085", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 176." - }, - { - "id": "11086", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 177." - }, - { - "id": "11087", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 178." - }, - { - "id": "11088", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "11089", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 180." - }, - { - "id": "11090", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 181." - }, - { - "id": "11091", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 182." - }, - { - "id": "11092", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 183." - }, - { - "id": "11093", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "11094", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 185." - }, - { - "id": "11095", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 186." - }, - { - "id": "11096", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 187." - }, - { - "id": "11097", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 188." - }, - { - "id": "11098", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "11099", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 190." - }, - { - "id": "11100", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 191." - }, - { - "id": "11101", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 192." - }, - { - "id": "11102", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 193." - }, - { - "id": "11103", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "11104", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 195." - }, - { - "id": "11105", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 196." - }, - { - "id": "11106", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 197." - }, - { - "id": "11107", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 198." - }, - { - "id": "11108", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "11109", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 200." - }, - { - "id": "11110", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 201." - }, - { - "id": "11111", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 202." - }, - { - "id": "11112", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 203." - }, - { - "id": "11113", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "11114", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 205." - }, - { - "id": "11115", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 206." - }, - { - "id": "11116", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 207." - }, - { - "id": "11117", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 208." - }, - { - "id": "11118", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "11119", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 210." - }, - { - "id": "11120", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 211." - }, - { - "id": "11121", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 212." - }, - { - "id": "11122", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 213." - }, - { - "id": "11123", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "11124", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 215." - }, - { - "id": "11125", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 216." - }, - { - "id": "11126", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 217." - }, - { - "id": "11127", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 218." - }, - { - "id": "11128", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "11129", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 220." - }, - { - "id": "11130", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 221." - }, - { - "id": "11131", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 222." - }, - { - "id": "11132", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 223." - }, - { - "id": "11133", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "11134", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 225." - }, - { - "id": "11135", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 226." - }, - { - "id": "11136", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 227." - }, - { - "id": "11137", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 228." - }, - { - "id": "11138", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "11139", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 230." - }, - { - "id": "11140", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 231." - }, - { - "id": "11141", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 232." - }, - { - "id": "11142", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 233." - }, - { - "id": "11143", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "11144", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 235." - }, - { - "id": "11145", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 236." - }, - { - "id": "11146", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 237." - }, - { - "id": "11147", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 238." - }, - { - "id": "11148", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "11149", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 240." - }, - { - "id": "11150", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 241." - }, - { - "id": "11151", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 242." - }, - { - "id": "11152", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 243." - }, - { - "id": "11153", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "11154", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 245." - }, - { - "id": "11155", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 246." - }, - { - "id": "11156", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 247." - }, - { - "id": "11157", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 248." - }, - { - "id": "11158", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "11159", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 250." - }, - { - "id": "11160", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 251." - }, - { - "id": "11161", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 252." - }, - { - "id": "11162", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 253." - }, - { - "id": "11163", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "11164", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 255." - }, - { - "id": "11165", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 256." - }, - { - "id": "11166", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 257." - }, - { - "id": "11167", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 258." - }, - { - "id": "11168", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "11169", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 260." - }, - { - "id": "11170", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 261." - }, - { - "id": "11171", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 262." - }, - { - "id": "11172", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 263." - }, - { - "id": "11173", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "11174", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 265." - }, - { - "id": "11175", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 266." - }, - { - "id": "11176", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 267." - }, - { - "id": "11177", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 268." - }, - { - "id": "11178", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "11179", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 270." - }, - { - "id": "11180", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 271." - }, - { - "id": "11181", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 272." - }, - { - "id": "11182", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 273." - }, - { - "id": "11183", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "11184", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 275." - }, - { - "id": "11185", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 276." - }, - { - "id": "11186", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 277." - }, - { - "id": "11187", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 278." - }, - { - "id": "11188", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "11189", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 280." - }, - { - "id": "11190", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 281." - }, - { - "id": "11191", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 282." - }, - { - "id": "11192", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 283." - }, - { - "id": "11193", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "11194", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 285." - }, - { - "id": "11195", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 286." - }, - { - "id": "11196", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 287." - }, - { - "id": "11197", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 288." - }, - { - "id": "11198", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "11199", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 290." - }, - { - "id": "11200", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 291." - }, - { - "id": "11201", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 292." - }, - { - "id": "11202", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 293." - }, - { - "id": "11203", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "11204", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund: agenda only; no decision yet. Thread 295." - }, - { - "id": "11205", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen invoice refund? Please add this question to the meeting. Thread 296." - }, - { - "id": "11206", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: lumen invoice refund. This empty template supplies no policy. Thread 297." - }, - { - "id": "11207", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "lumen status digest. invoice is a document tag. refund is a separate training exercise. No answer. Thread 298." - }, - { - "id": "11208", - "chatId": "104", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old lumen invoice refund proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "11209", - "chatId": "105", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Harbor release freeze decision: stop deployments until the database check passes." - }, - { - "id": "11210", - "chatId": "105", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "The Harbor release freeze owner is Oren; the decision is recorded above." - }, - { - "id": "11211", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 0." - }, - { - "id": "11212", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 1." - }, - { - "id": "11213", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 2." - }, - { - "id": "11214", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 3." - }, - { - "id": "11215", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "11216", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 5." - }, - { - "id": "11217", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 6." - }, - { - "id": "11218", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 7." - }, - { - "id": "11219", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 8." - }, - { - "id": "11220", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "11221", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 10." - }, - { - "id": "11222", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 11." - }, - { - "id": "11223", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 12." - }, - { - "id": "11224", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 13." - }, - { - "id": "11225", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "11226", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 15." - }, - { - "id": "11227", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 16." - }, - { - "id": "11228", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 17." - }, - { - "id": "11229", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 18." - }, - { - "id": "11230", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "11231", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 20." - }, - { - "id": "11232", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 21." - }, - { - "id": "11233", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 22." - }, - { - "id": "11234", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 23." - }, - { - "id": "11235", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "11236", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 25." - }, - { - "id": "11237", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 26." - }, - { - "id": "11238", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 27." - }, - { - "id": "11239", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 28." - }, - { - "id": "11240", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "11241", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 30." - }, - { - "id": "11242", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 31." - }, - { - "id": "11243", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 32." - }, - { - "id": "11244", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 33." - }, - { - "id": "11245", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "11246", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 35." - }, - { - "id": "11247", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 36." - }, - { - "id": "11248", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 37." - }, - { - "id": "11249", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 38." - }, - { - "id": "11250", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "11251", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 40." - }, - { - "id": "11252", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 41." - }, - { - "id": "11253", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 42." - }, - { - "id": "11254", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 43." - }, - { - "id": "11255", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "11256", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 45." - }, - { - "id": "11257", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 46." - }, - { - "id": "11258", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 47." - }, - { - "id": "11259", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 48." - }, - { - "id": "11260", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "11261", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 50." - }, - { - "id": "11262", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 51." - }, - { - "id": "11263", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 52." - }, - { - "id": "11264", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 53." - }, - { - "id": "11265", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "11266", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 55." - }, - { - "id": "11267", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 56." - }, - { - "id": "11268", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 57." - }, - { - "id": "11269", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 58." - }, - { - "id": "11270", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "11271", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 60." - }, - { - "id": "11272", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 61." - }, - { - "id": "11273", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 62." - }, - { - "id": "11274", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 63." - }, - { - "id": "11275", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "11276", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 65." - }, - { - "id": "11277", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 66." - }, - { - "id": "11278", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 67." - }, - { - "id": "11279", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 68." - }, - { - "id": "11280", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "11281", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 70." - }, - { - "id": "11282", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 71." - }, - { - "id": "11283", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 72." - }, - { - "id": "11284", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 73." - }, - { - "id": "11285", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "11286", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 75." - }, - { - "id": "11287", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 76." - }, - { - "id": "11288", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 77." - }, - { - "id": "11289", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 78." - }, - { - "id": "11290", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "11291", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 80." - }, - { - "id": "11292", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 81." - }, - { - "id": "11293", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 82." - }, - { - "id": "11294", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 83." - }, - { - "id": "11295", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "11296", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 85." - }, - { - "id": "11297", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 86." - }, - { - "id": "11298", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 87." - }, - { - "id": "11299", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 88." - }, - { - "id": "11300", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "11301", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 90." - }, - { - "id": "11302", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 91." - }, - { - "id": "11303", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 92." - }, - { - "id": "11304", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 93." - }, - { - "id": "11305", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "11306", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 95." - }, - { - "id": "11307", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 96." - }, - { - "id": "11308", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 97." - }, - { - "id": "11309", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 98." - }, - { - "id": "11310", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "11311", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 100." - }, - { - "id": "11312", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 101." - }, - { - "id": "11313", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 102." - }, - { - "id": "11314", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 103." - }, - { - "id": "11315", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "11316", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 105." - }, - { - "id": "11317", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 106." - }, - { - "id": "11318", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 107." - }, - { - "id": "11319", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 108." - }, - { - "id": "11320", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "11321", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 110." - }, - { - "id": "11322", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 111." - }, - { - "id": "11323", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 112." - }, - { - "id": "11324", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 113." - }, - { - "id": "11325", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "11326", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 115." - }, - { - "id": "11327", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 116." - }, - { - "id": "11328", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 117." - }, - { - "id": "11329", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 118." - }, - { - "id": "11330", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "11331", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 120." - }, - { - "id": "11332", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 121." - }, - { - "id": "11333", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 122." - }, - { - "id": "11334", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 123." - }, - { - "id": "11335", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "11336", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 125." - }, - { - "id": "11337", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 126." - }, - { - "id": "11338", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 127." - }, - { - "id": "11339", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 128." - }, - { - "id": "11340", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "11341", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 130." - }, - { - "id": "11342", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 131." - }, - { - "id": "11343", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 132." - }, - { - "id": "11344", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 133." - }, - { - "id": "11345", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "11346", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 135." - }, - { - "id": "11347", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 136." - }, - { - "id": "11348", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 137." - }, - { - "id": "11349", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 138." - }, - { - "id": "11350", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "11351", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 140." - }, - { - "id": "11352", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 141." - }, - { - "id": "11353", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 142." - }, - { - "id": "11354", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 143." - }, - { - "id": "11355", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "11356", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 145." - }, - { - "id": "11357", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 146." - }, - { - "id": "11358", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 147." - }, - { - "id": "11359", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 148." - }, - { - "id": "11360", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "11361", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 150." - }, - { - "id": "11362", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 151." - }, - { - "id": "11363", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 152." - }, - { - "id": "11364", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 153." - }, - { - "id": "11365", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "11366", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 155." - }, - { - "id": "11367", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 156." - }, - { - "id": "11368", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 157." - }, - { - "id": "11369", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 158." - }, - { - "id": "11370", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "11371", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 160." - }, - { - "id": "11372", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 161." - }, - { - "id": "11373", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 162." - }, - { - "id": "11374", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 163." - }, - { - "id": "11375", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "11376", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 165." - }, - { - "id": "11377", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 166." - }, - { - "id": "11378", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 167." - }, - { - "id": "11379", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 168." - }, - { - "id": "11380", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "11381", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 170." - }, - { - "id": "11382", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 171." - }, - { - "id": "11383", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 172." - }, - { - "id": "11384", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 173." - }, - { - "id": "11385", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "11386", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 175." - }, - { - "id": "11387", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 176." - }, - { - "id": "11388", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 177." - }, - { - "id": "11389", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 178." - }, - { - "id": "11390", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "11391", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 180." - }, - { - "id": "11392", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 181." - }, - { - "id": "11393", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 182." - }, - { - "id": "11394", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 183." - }, - { - "id": "11395", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "11396", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 185." - }, - { - "id": "11397", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 186." - }, - { - "id": "11398", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 187." - }, - { - "id": "11399", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 188." - }, - { - "id": "11400", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "11401", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 190." - }, - { - "id": "11402", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 191." - }, - { - "id": "11403", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 192." - }, - { - "id": "11404", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 193." - }, - { - "id": "11405", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "11406", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 195." - }, - { - "id": "11407", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 196." - }, - { - "id": "11408", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 197." - }, - { - "id": "11409", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 198." - }, - { - "id": "11410", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "11411", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 200." - }, - { - "id": "11412", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 201." - }, - { - "id": "11413", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 202." - }, - { - "id": "11414", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 203." - }, - { - "id": "11415", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "11416", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 205." - }, - { - "id": "11417", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 206." - }, - { - "id": "11418", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 207." - }, - { - "id": "11419", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 208." - }, - { - "id": "11420", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "11421", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 210." - }, - { - "id": "11422", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 211." - }, - { - "id": "11423", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 212." - }, - { - "id": "11424", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 213." - }, - { - "id": "11425", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "11426", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 215." - }, - { - "id": "11427", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 216." - }, - { - "id": "11428", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 217." - }, - { - "id": "11429", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 218." - }, - { - "id": "11430", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "11431", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 220." - }, - { - "id": "11432", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 221." - }, - { - "id": "11433", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 222." - }, - { - "id": "11434", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 223." - }, - { - "id": "11435", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "11436", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 225." - }, - { - "id": "11437", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 226." - }, - { - "id": "11438", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 227." - }, - { - "id": "11439", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 228." - }, - { - "id": "11440", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "11441", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 230." - }, - { - "id": "11442", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 231." - }, - { - "id": "11443", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 232." - }, - { - "id": "11444", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 233." - }, - { - "id": "11445", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "11446", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 235." - }, - { - "id": "11447", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 236." - }, - { - "id": "11448", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 237." - }, - { - "id": "11449", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 238." - }, - { - "id": "11450", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "11451", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 240." - }, - { - "id": "11452", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 241." - }, - { - "id": "11453", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 242." - }, - { - "id": "11454", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 243." - }, - { - "id": "11455", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "11456", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 245." - }, - { - "id": "11457", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 246." - }, - { - "id": "11458", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 247." - }, - { - "id": "11459", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 248." - }, - { - "id": "11460", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "11461", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 250." - }, - { - "id": "11462", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 251." - }, - { - "id": "11463", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 252." - }, - { - "id": "11464", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 253." - }, - { - "id": "11465", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "11466", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 255." - }, - { - "id": "11467", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 256." - }, - { - "id": "11468", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 257." - }, - { - "id": "11469", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 258." - }, - { - "id": "11470", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "11471", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 260." - }, - { - "id": "11472", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 261." - }, - { - "id": "11473", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 262." - }, - { - "id": "11474", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 263." - }, - { - "id": "11475", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "11476", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 265." - }, - { - "id": "11477", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 266." - }, - { - "id": "11478", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 267." - }, - { - "id": "11479", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 268." - }, - { - "id": "11480", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "11481", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 270." - }, - { - "id": "11482", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 271." - }, - { - "id": "11483", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 272." - }, - { - "id": "11484", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 273." - }, - { - "id": "11485", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "11486", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 275." - }, - { - "id": "11487", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 276." - }, - { - "id": "11488", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 277." - }, - { - "id": "11489", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 278." - }, - { - "id": "11490", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "11491", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 280." - }, - { - "id": "11492", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 281." - }, - { - "id": "11493", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 282." - }, - { - "id": "11494", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 283." - }, - { - "id": "11495", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "11496", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 285." - }, - { - "id": "11497", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 286." - }, - { - "id": "11498", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 287." - }, - { - "id": "11499", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 288." - }, - { - "id": "11500", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "11501", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 290." - }, - { - "id": "11502", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 291." - }, - { - "id": "11503", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 292." - }, - { - "id": "11504", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 293." - }, - { - "id": "11505", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "11506", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze: agenda only; no decision yet. Thread 295." - }, - { - "id": "11507", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor release freeze? Please add this question to the meeting. Thread 296." - }, - { - "id": "11508", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: harbor release freeze. This empty template supplies no policy. Thread 297." - }, - { - "id": "11509", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "harbor status digest. release is a document tag. freeze is a separate training exercise. No answer. Thread 298." - }, - { - "id": "11510", - "chatId": "105", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old harbor release freeze proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "11511", - "chatId": "106", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Maple archive restore decision: recover the March snapshot into a separate workspace." - }, - { - "id": "11512", - "chatId": "106", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "The Maple archive restore owner is Ada; the decision is recorded above." - }, - { - "id": "11513", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 0." - }, - { - "id": "11514", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 1." - }, - { - "id": "11515", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 2." - }, - { - "id": "11516", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 3." - }, - { - "id": "11517", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "11518", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 5." - }, - { - "id": "11519", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 6." - }, - { - "id": "11520", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 7." - }, - { - "id": "11521", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 8." - }, - { - "id": "11522", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "11523", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 10." - }, - { - "id": "11524", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 11." - }, - { - "id": "11525", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 12." - }, - { - "id": "11526", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 13." - }, - { - "id": "11527", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "11528", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 15." - }, - { - "id": "11529", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 16." - }, - { - "id": "11530", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 17." - }, - { - "id": "11531", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 18." - }, - { - "id": "11532", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "11533", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 20." - }, - { - "id": "11534", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 21." - }, - { - "id": "11535", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 22." - }, - { - "id": "11536", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 23." - }, - { - "id": "11537", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "11538", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 25." - }, - { - "id": "11539", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 26." - }, - { - "id": "11540", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 27." - }, - { - "id": "11541", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 28." - }, - { - "id": "11542", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "11543", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 30." - }, - { - "id": "11544", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 31." - }, - { - "id": "11545", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 32." - }, - { - "id": "11546", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 33." - }, - { - "id": "11547", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "11548", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 35." - }, - { - "id": "11549", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 36." - }, - { - "id": "11550", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 37." - }, - { - "id": "11551", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 38." - }, - { - "id": "11552", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "11553", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 40." - }, - { - "id": "11554", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 41." - }, - { - "id": "11555", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 42." - }, - { - "id": "11556", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 43." - }, - { - "id": "11557", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "11558", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 45." - }, - { - "id": "11559", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 46." - }, - { - "id": "11560", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 47." - }, - { - "id": "11561", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 48." - }, - { - "id": "11562", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "11563", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 50." - }, - { - "id": "11564", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 51." - }, - { - "id": "11565", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 52." - }, - { - "id": "11566", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 53." - }, - { - "id": "11567", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "11568", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 55." - }, - { - "id": "11569", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 56." - }, - { - "id": "11570", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 57." - }, - { - "id": "11571", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 58." - }, - { - "id": "11572", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "11573", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 60." - }, - { - "id": "11574", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 61." - }, - { - "id": "11575", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 62." - }, - { - "id": "11576", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 63." - }, - { - "id": "11577", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "11578", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 65." - }, - { - "id": "11579", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 66." - }, - { - "id": "11580", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 67." - }, - { - "id": "11581", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 68." - }, - { - "id": "11582", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "11583", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 70." - }, - { - "id": "11584", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 71." - }, - { - "id": "11585", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 72." - }, - { - "id": "11586", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 73." - }, - { - "id": "11587", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "11588", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 75." - }, - { - "id": "11589", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 76." - }, - { - "id": "11590", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 77." - }, - { - "id": "11591", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 78." - }, - { - "id": "11592", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "11593", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 80." - }, - { - "id": "11594", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 81." - }, - { - "id": "11595", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 82." - }, - { - "id": "11596", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 83." - }, - { - "id": "11597", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "11598", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 85." - }, - { - "id": "11599", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 86." - }, - { - "id": "11600", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 87." - }, - { - "id": "11601", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 88." - }, - { - "id": "11602", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "11603", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 90." - }, - { - "id": "11604", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 91." - }, - { - "id": "11605", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 92." - }, - { - "id": "11606", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 93." - }, - { - "id": "11607", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "11608", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 95." - }, - { - "id": "11609", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 96." - }, - { - "id": "11610", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 97." - }, - { - "id": "11611", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 98." - }, - { - "id": "11612", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "11613", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 100." - }, - { - "id": "11614", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 101." - }, - { - "id": "11615", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 102." - }, - { - "id": "11616", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 103." - }, - { - "id": "11617", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "11618", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 105." - }, - { - "id": "11619", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 106." - }, - { - "id": "11620", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 107." - }, - { - "id": "11621", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 108." - }, - { - "id": "11622", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "11623", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 110." - }, - { - "id": "11624", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 111." - }, - { - "id": "11625", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 112." - }, - { - "id": "11626", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 113." - }, - { - "id": "11627", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "11628", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 115." - }, - { - "id": "11629", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 116." - }, - { - "id": "11630", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 117." - }, - { - "id": "11631", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 118." - }, - { - "id": "11632", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "11633", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 120." - }, - { - "id": "11634", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 121." - }, - { - "id": "11635", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 122." - }, - { - "id": "11636", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 123." - }, - { - "id": "11637", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "11638", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 125." - }, - { - "id": "11639", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 126." - }, - { - "id": "11640", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 127." - }, - { - "id": "11641", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 128." - }, - { - "id": "11642", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "11643", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 130." - }, - { - "id": "11644", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 131." - }, - { - "id": "11645", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 132." - }, - { - "id": "11646", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 133." - }, - { - "id": "11647", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "11648", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 135." - }, - { - "id": "11649", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 136." - }, - { - "id": "11650", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 137." - }, - { - "id": "11651", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 138." - }, - { - "id": "11652", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "11653", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 140." - }, - { - "id": "11654", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 141." - }, - { - "id": "11655", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 142." - }, - { - "id": "11656", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 143." - }, - { - "id": "11657", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "11658", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 145." - }, - { - "id": "11659", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 146." - }, - { - "id": "11660", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 147." - }, - { - "id": "11661", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 148." - }, - { - "id": "11662", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "11663", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 150." - }, - { - "id": "11664", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 151." - }, - { - "id": "11665", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 152." - }, - { - "id": "11666", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 153." - }, - { - "id": "11667", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "11668", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 155." - }, - { - "id": "11669", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 156." - }, - { - "id": "11670", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 157." - }, - { - "id": "11671", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 158." - }, - { - "id": "11672", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "11673", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 160." - }, - { - "id": "11674", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 161." - }, - { - "id": "11675", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 162." - }, - { - "id": "11676", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 163." - }, - { - "id": "11677", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "11678", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 165." - }, - { - "id": "11679", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 166." - }, - { - "id": "11680", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 167." - }, - { - "id": "11681", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 168." - }, - { - "id": "11682", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "11683", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 170." - }, - { - "id": "11684", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 171." - }, - { - "id": "11685", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 172." - }, - { - "id": "11686", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 173." - }, - { - "id": "11687", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "11688", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 175." - }, - { - "id": "11689", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 176." - }, - { - "id": "11690", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 177." - }, - { - "id": "11691", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 178." - }, - { - "id": "11692", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "11693", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 180." - }, - { - "id": "11694", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 181." - }, - { - "id": "11695", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 182." - }, - { - "id": "11696", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 183." - }, - { - "id": "11697", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "11698", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 185." - }, - { - "id": "11699", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 186." - }, - { - "id": "11700", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 187." - }, - { - "id": "11701", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 188." - }, - { - "id": "11702", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "11703", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 190." - }, - { - "id": "11704", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 191." - }, - { - "id": "11705", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 192." - }, - { - "id": "11706", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 193." - }, - { - "id": "11707", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "11708", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 195." - }, - { - "id": "11709", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 196." - }, - { - "id": "11710", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 197." - }, - { - "id": "11711", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 198." - }, - { - "id": "11712", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "11713", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 200." - }, - { - "id": "11714", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 201." - }, - { - "id": "11715", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 202." - }, - { - "id": "11716", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 203." - }, - { - "id": "11717", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "11718", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 205." - }, - { - "id": "11719", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 206." - }, - { - "id": "11720", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 207." - }, - { - "id": "11721", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 208." - }, - { - "id": "11722", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "11723", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 210." - }, - { - "id": "11724", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 211." - }, - { - "id": "11725", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 212." - }, - { - "id": "11726", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 213." - }, - { - "id": "11727", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "11728", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 215." - }, - { - "id": "11729", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 216." - }, - { - "id": "11730", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 217." - }, - { - "id": "11731", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 218." - }, - { - "id": "11732", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "11733", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 220." - }, - { - "id": "11734", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 221." - }, - { - "id": "11735", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 222." - }, - { - "id": "11736", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 223." - }, - { - "id": "11737", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "11738", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 225." - }, - { - "id": "11739", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 226." - }, - { - "id": "11740", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 227." - }, - { - "id": "11741", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 228." - }, - { - "id": "11742", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "11743", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 230." - }, - { - "id": "11744", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 231." - }, - { - "id": "11745", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 232." - }, - { - "id": "11746", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 233." - }, - { - "id": "11747", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "11748", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 235." - }, - { - "id": "11749", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 236." - }, - { - "id": "11750", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 237." - }, - { - "id": "11751", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 238." - }, - { - "id": "11752", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "11753", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 240." - }, - { - "id": "11754", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 241." - }, - { - "id": "11755", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 242." - }, - { - "id": "11756", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 243." - }, - { - "id": "11757", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "11758", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 245." - }, - { - "id": "11759", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 246." - }, - { - "id": "11760", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 247." - }, - { - "id": "11761", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 248." - }, - { - "id": "11762", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "11763", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 250." - }, - { - "id": "11764", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 251." - }, - { - "id": "11765", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 252." - }, - { - "id": "11766", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 253." - }, - { - "id": "11767", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "11768", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 255." - }, - { - "id": "11769", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 256." - }, - { - "id": "11770", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 257." - }, - { - "id": "11771", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 258." - }, - { - "id": "11772", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "11773", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 260." - }, - { - "id": "11774", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 261." - }, - { - "id": "11775", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 262." - }, - { - "id": "11776", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 263." - }, - { - "id": "11777", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "11778", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 265." - }, - { - "id": "11779", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 266." - }, - { - "id": "11780", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 267." - }, - { - "id": "11781", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 268." - }, - { - "id": "11782", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "11783", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 270." - }, - { - "id": "11784", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 271." - }, - { - "id": "11785", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 272." - }, - { - "id": "11786", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 273." - }, - { - "id": "11787", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "11788", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 275." - }, - { - "id": "11789", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 276." - }, - { - "id": "11790", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 277." - }, - { - "id": "11791", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 278." - }, - { - "id": "11792", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "11793", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 280." - }, - { - "id": "11794", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 281." - }, - { - "id": "11795", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 282." - }, - { - "id": "11796", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 283." - }, - { - "id": "11797", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "11798", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 285." - }, - { - "id": "11799", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 286." - }, - { - "id": "11800", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 287." - }, - { - "id": "11801", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 288." - }, - { - "id": "11802", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "11803", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 290." - }, - { - "id": "11804", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 291." - }, - { - "id": "11805", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 292." - }, - { - "id": "11806", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 293." - }, - { - "id": "11807", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "11808", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore: agenda only; no decision yet. Thread 295." - }, - { - "id": "11809", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple archive restore? Please add this question to the meeting. Thread 296." - }, - { - "id": "11810", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: maple archive restore. This empty template supplies no policy. Thread 297." - }, - { - "id": "11811", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "maple status digest. archive is a document tag. restore is a separate training exercise. No answer. Thread 298." - }, - { - "id": "11812", - "chatId": "106", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old maple archive restore proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "11813", - "chatId": "107", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Вектор миграция отмена: отключить задание и вернуть схему версии 8." - }, - { - "id": "11814", - "chatId": "107", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Вектор миграция отмена проверена, владелец Лев." - }, - { - "id": "11815", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 0." - }, - { - "id": "11816", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 1." - }, - { - "id": "11817", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 2." - }, - { - "id": "11818", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 3." - }, - { - "id": "11819", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "11820", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 5." - }, - { - "id": "11821", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 6." - }, - { - "id": "11822", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 7." - }, - { - "id": "11823", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 8." - }, - { - "id": "11824", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "11825", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 10." - }, - { - "id": "11826", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 11." - }, - { - "id": "11827", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 12." - }, - { - "id": "11828", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 13." - }, - { - "id": "11829", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "11830", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 15." - }, - { - "id": "11831", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 16." - }, - { - "id": "11832", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 17." - }, - { - "id": "11833", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 18." - }, - { - "id": "11834", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "11835", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 20." - }, - { - "id": "11836", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 21." - }, - { - "id": "11837", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 22." - }, - { - "id": "11838", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 23." - }, - { - "id": "11839", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "11840", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 25." - }, - { - "id": "11841", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 26." - }, - { - "id": "11842", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 27." - }, - { - "id": "11843", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 28." - }, - { - "id": "11844", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "11845", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 30." - }, - { - "id": "11846", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 31." - }, - { - "id": "11847", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 32." - }, - { - "id": "11848", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 33." - }, - { - "id": "11849", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "11850", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 35." - }, - { - "id": "11851", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 36." - }, - { - "id": "11852", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 37." - }, - { - "id": "11853", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 38." - }, - { - "id": "11854", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "11855", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 40." - }, - { - "id": "11856", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 41." - }, - { - "id": "11857", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 42." - }, - { - "id": "11858", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 43." - }, - { - "id": "11859", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "11860", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 45." - }, - { - "id": "11861", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 46." - }, - { - "id": "11862", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 47." - }, - { - "id": "11863", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 48." - }, - { - "id": "11864", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "11865", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 50." - }, - { - "id": "11866", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 51." - }, - { - "id": "11867", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 52." - }, - { - "id": "11868", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 53." - }, - { - "id": "11869", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "11870", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 55." - }, - { - "id": "11871", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 56." - }, - { - "id": "11872", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 57." - }, - { - "id": "11873", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 58." - }, - { - "id": "11874", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "11875", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 60." - }, - { - "id": "11876", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 61." - }, - { - "id": "11877", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 62." - }, - { - "id": "11878", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 63." - }, - { - "id": "11879", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "11880", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 65." - }, - { - "id": "11881", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 66." - }, - { - "id": "11882", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 67." - }, - { - "id": "11883", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 68." - }, - { - "id": "11884", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "11885", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 70." - }, - { - "id": "11886", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 71." - }, - { - "id": "11887", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 72." - }, - { - "id": "11888", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 73." - }, - { - "id": "11889", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "11890", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 75." - }, - { - "id": "11891", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 76." - }, - { - "id": "11892", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 77." - }, - { - "id": "11893", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 78." - }, - { - "id": "11894", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "11895", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 80." - }, - { - "id": "11896", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 81." - }, - { - "id": "11897", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 82." - }, - { - "id": "11898", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 83." - }, - { - "id": "11899", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "11900", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 85." - }, - { - "id": "11901", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 86." - }, - { - "id": "11902", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 87." - }, - { - "id": "11903", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 88." - }, - { - "id": "11904", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "11905", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 90." - }, - { - "id": "11906", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 91." - }, - { - "id": "11907", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 92." - }, - { - "id": "11908", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 93." - }, - { - "id": "11909", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "11910", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 95." - }, - { - "id": "11911", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 96." - }, - { - "id": "11912", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 97." - }, - { - "id": "11913", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 98." - }, - { - "id": "11914", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "11915", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 100." - }, - { - "id": "11916", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 101." - }, - { - "id": "11917", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 102." - }, - { - "id": "11918", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 103." - }, - { - "id": "11919", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "11920", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 105." - }, - { - "id": "11921", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 106." - }, - { - "id": "11922", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 107." - }, - { - "id": "11923", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 108." - }, - { - "id": "11924", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "11925", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 110." - }, - { - "id": "11926", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 111." - }, - { - "id": "11927", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 112." - }, - { - "id": "11928", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 113." - }, - { - "id": "11929", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "11930", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 115." - }, - { - "id": "11931", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 116." - }, - { - "id": "11932", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 117." - }, - { - "id": "11933", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 118." - }, - { - "id": "11934", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "11935", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 120." - }, - { - "id": "11936", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 121." - }, - { - "id": "11937", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 122." - }, - { - "id": "11938", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 123." - }, - { - "id": "11939", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "11940", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 125." - }, - { - "id": "11941", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 126." - }, - { - "id": "11942", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 127." - }, - { - "id": "11943", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 128." - }, - { - "id": "11944", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "11945", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 130." - }, - { - "id": "11946", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 131." - }, - { - "id": "11947", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 132." - }, - { - "id": "11948", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 133." - }, - { - "id": "11949", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "11950", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 135." - }, - { - "id": "11951", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 136." - }, - { - "id": "11952", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 137." - }, - { - "id": "11953", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 138." - }, - { - "id": "11954", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "11955", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 140." - }, - { - "id": "11956", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 141." - }, - { - "id": "11957", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 142." - }, - { - "id": "11958", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 143." - }, - { - "id": "11959", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "11960", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 145." - }, - { - "id": "11961", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 146." - }, - { - "id": "11962", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 147." - }, - { - "id": "11963", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 148." - }, - { - "id": "11964", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "11965", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 150." - }, - { - "id": "11966", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 151." - }, - { - "id": "11967", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 152." - }, - { - "id": "11968", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 153." - }, - { - "id": "11969", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "11970", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 155." - }, - { - "id": "11971", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 156." - }, - { - "id": "11972", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 157." - }, - { - "id": "11973", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 158." - }, - { - "id": "11974", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "11975", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 160." - }, - { - "id": "11976", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 161." - }, - { - "id": "11977", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 162." - }, - { - "id": "11978", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 163." - }, - { - "id": "11979", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "11980", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 165." - }, - { - "id": "11981", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 166." - }, - { - "id": "11982", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 167." - }, - { - "id": "11983", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 168." - }, - { - "id": "11984", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "11985", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 170." - }, - { - "id": "11986", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 171." - }, - { - "id": "11987", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 172." - }, - { - "id": "11988", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 173." - }, - { - "id": "11989", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "11990", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 175." - }, - { - "id": "11991", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 176." - }, - { - "id": "11992", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 177." - }, - { - "id": "11993", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 178." - }, - { - "id": "11994", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "11995", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 180." - }, - { - "id": "11996", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 181." - }, - { - "id": "11997", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 182." - }, - { - "id": "11998", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 183." - }, - { - "id": "11999", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "12000", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 185." - }, - { - "id": "12001", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 186." - }, - { - "id": "12002", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 187." - }, - { - "id": "12003", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 188." - }, - { - "id": "12004", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "12005", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 190." - }, - { - "id": "12006", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 191." - }, - { - "id": "12007", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 192." - }, - { - "id": "12008", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 193." - }, - { - "id": "12009", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "12010", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 195." - }, - { - "id": "12011", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 196." - }, - { - "id": "12012", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 197." - }, - { - "id": "12013", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 198." - }, - { - "id": "12014", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "12015", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 200." - }, - { - "id": "12016", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 201." - }, - { - "id": "12017", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 202." - }, - { - "id": "12018", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 203." - }, - { - "id": "12019", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "12020", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 205." - }, - { - "id": "12021", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 206." - }, - { - "id": "12022", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 207." - }, - { - "id": "12023", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 208." - }, - { - "id": "12024", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "12025", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 210." - }, - { - "id": "12026", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 211." - }, - { - "id": "12027", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 212." - }, - { - "id": "12028", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 213." - }, - { - "id": "12029", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "12030", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 215." - }, - { - "id": "12031", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 216." - }, - { - "id": "12032", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 217." - }, - { - "id": "12033", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 218." - }, - { - "id": "12034", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "12035", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 220." - }, - { - "id": "12036", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 221." - }, - { - "id": "12037", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 222." - }, - { - "id": "12038", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 223." - }, - { - "id": "12039", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "12040", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 225." - }, - { - "id": "12041", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 226." - }, - { - "id": "12042", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 227." - }, - { - "id": "12043", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 228." - }, - { - "id": "12044", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "12045", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 230." - }, - { - "id": "12046", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 231." - }, - { - "id": "12047", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 232." - }, - { - "id": "12048", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 233." - }, - { - "id": "12049", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "12050", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 235." - }, - { - "id": "12051", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 236." - }, - { - "id": "12052", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 237." - }, - { - "id": "12053", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 238." - }, - { - "id": "12054", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "12055", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 240." - }, - { - "id": "12056", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 241." - }, - { - "id": "12057", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 242." - }, - { - "id": "12058", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 243." - }, - { - "id": "12059", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "12060", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 245." - }, - { - "id": "12061", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 246." - }, - { - "id": "12062", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 247." - }, - { - "id": "12063", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 248." - }, - { - "id": "12064", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "12065", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 250." - }, - { - "id": "12066", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 251." - }, - { - "id": "12067", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 252." - }, - { - "id": "12068", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 253." - }, - { - "id": "12069", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "12070", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 255." - }, - { - "id": "12071", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 256." - }, - { - "id": "12072", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 257." - }, - { - "id": "12073", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 258." - }, - { - "id": "12074", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "12075", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 260." - }, - { - "id": "12076", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 261." - }, - { - "id": "12077", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 262." - }, - { - "id": "12078", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 263." - }, - { - "id": "12079", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "12080", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 265." - }, - { - "id": "12081", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 266." - }, - { - "id": "12082", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 267." - }, - { - "id": "12083", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 268." - }, - { - "id": "12084", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "12085", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 270." - }, - { - "id": "12086", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 271." - }, - { - "id": "12087", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 272." - }, - { - "id": "12088", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 273." - }, - { - "id": "12089", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "12090", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 275." - }, - { - "id": "12091", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 276." - }, - { - "id": "12092", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 277." - }, - { - "id": "12093", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 278." - }, - { - "id": "12094", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "12095", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 280." - }, - { - "id": "12096", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 281." - }, - { - "id": "12097", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 282." - }, - { - "id": "12098", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 283." - }, - { - "id": "12099", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "12100", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 285." - }, - { - "id": "12101", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 286." - }, - { - "id": "12102", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 287." - }, - { - "id": "12103", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 288." - }, - { - "id": "12104", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "12105", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 290." - }, - { - "id": "12106", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 291." - }, - { - "id": "12107", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 292." - }, - { - "id": "12108", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 293." - }, - { - "id": "12109", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "12110", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена: agenda only; no decision yet. Thread 295." - }, - { - "id": "12111", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор миграция отмена? Please add this question to the meeting. Thread 296." - }, - { - "id": "12112", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: вектор миграция отмена. This empty template supplies no policy. Thread 297." - }, - { - "id": "12113", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "вектор status digest. миграция is a document tag. отмена is a separate training exercise. No answer. Thread 298." - }, - { - "id": "12114", - "chatId": "107", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old вектор миграция отмена proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "12115", - "chatId": "108", - "senderId": "700", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "Prism renewal discount approved: apply 15 percent to the annual contract." - }, - { - "id": "12116", - "chatId": "108", - "senderId": "701", - "timestamp": "2026-10-02T12:00:00.000Z", - "text": "The Prism renewal discount owner is Tess; approval is recorded above." - }, - { - "id": "12117", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 0." - }, - { - "id": "12118", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 1." - }, - { - "id": "12119", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 2." - }, - { - "id": "12120", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 3." - }, - { - "id": "12121", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 4." - }, - { - "id": "12122", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 5." - }, - { - "id": "12123", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 6." - }, - { - "id": "12124", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 7." - }, - { - "id": "12125", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 8." - }, - { - "id": "12126", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 9." - }, - { - "id": "12127", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 10." - }, - { - "id": "12128", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 11." - }, - { - "id": "12129", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 12." - }, - { - "id": "12130", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 13." - }, - { - "id": "12131", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 14." - }, - { - "id": "12132", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 15." - }, - { - "id": "12133", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 16." - }, - { - "id": "12134", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 17." - }, - { - "id": "12135", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 18." - }, - { - "id": "12136", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 19." - }, - { - "id": "12137", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 20." - }, - { - "id": "12138", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 21." - }, - { - "id": "12139", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 22." - }, - { - "id": "12140", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 23." - }, - { - "id": "12141", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 24." - }, - { - "id": "12142", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 25." - }, - { - "id": "12143", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 26." - }, - { - "id": "12144", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 27." - }, - { - "id": "12145", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 28." - }, - { - "id": "12146", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 29." - }, - { - "id": "12147", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 30." - }, - { - "id": "12148", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 31." - }, - { - "id": "12149", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 32." - }, - { - "id": "12150", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 33." - }, - { - "id": "12151", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 34." - }, - { - "id": "12152", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 35." - }, - { - "id": "12153", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 36." - }, - { - "id": "12154", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 37." - }, - { - "id": "12155", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 38." - }, - { - "id": "12156", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 39." - }, - { - "id": "12157", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 40." - }, - { - "id": "12158", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 41." - }, - { - "id": "12159", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 42." - }, - { - "id": "12160", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 43." - }, - { - "id": "12161", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 44." - }, - { - "id": "12162", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 45." - }, - { - "id": "12163", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 46." - }, - { - "id": "12164", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 47." - }, - { - "id": "12165", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 48." - }, - { - "id": "12166", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 49." - }, - { - "id": "12167", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 50." - }, - { - "id": "12168", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 51." - }, - { - "id": "12169", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 52." - }, - { - "id": "12170", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 53." - }, - { - "id": "12171", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 54." - }, - { - "id": "12172", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 55." - }, - { - "id": "12173", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 56." - }, - { - "id": "12174", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 57." - }, - { - "id": "12175", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 58." - }, - { - "id": "12176", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 59." - }, - { - "id": "12177", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 60." - }, - { - "id": "12178", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 61." - }, - { - "id": "12179", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 62." - }, - { - "id": "12180", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 63." - }, - { - "id": "12181", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 64." - }, - { - "id": "12182", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 65." - }, - { - "id": "12183", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 66." - }, - { - "id": "12184", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 67." - }, - { - "id": "12185", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 68." - }, - { - "id": "12186", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 69." - }, - { - "id": "12187", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 70." - }, - { - "id": "12188", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 71." - }, - { - "id": "12189", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 72." - }, - { - "id": "12190", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 73." - }, - { - "id": "12191", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 74." - }, - { - "id": "12192", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 75." - }, - { - "id": "12193", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 76." - }, - { - "id": "12194", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 77." - }, - { - "id": "12195", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 78." - }, - { - "id": "12196", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 79." - }, - { - "id": "12197", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 80." - }, - { - "id": "12198", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 81." - }, - { - "id": "12199", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 82." - }, - { - "id": "12200", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 83." - }, - { - "id": "12201", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 84." - }, - { - "id": "12202", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 85." - }, - { - "id": "12203", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 86." - }, - { - "id": "12204", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 87." - }, - { - "id": "12205", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 88." - }, - { - "id": "12206", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 89." - }, - { - "id": "12207", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 90." - }, - { - "id": "12208", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 91." - }, - { - "id": "12209", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 92." - }, - { - "id": "12210", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 93." - }, - { - "id": "12211", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 94." - }, - { - "id": "12212", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 95." - }, - { - "id": "12213", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 96." - }, - { - "id": "12214", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 97." - }, - { - "id": "12215", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 98." - }, - { - "id": "12216", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 99." - }, - { - "id": "12217", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 100." - }, - { - "id": "12218", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 101." - }, - { - "id": "12219", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 102." - }, - { - "id": "12220", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 103." - }, - { - "id": "12221", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 104." - }, - { - "id": "12222", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 105." - }, - { - "id": "12223", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 106." - }, - { - "id": "12224", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 107." - }, - { - "id": "12225", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 108." - }, - { - "id": "12226", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 109." - }, - { - "id": "12227", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 110." - }, - { - "id": "12228", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 111." - }, - { - "id": "12229", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 112." - }, - { - "id": "12230", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 113." - }, - { - "id": "12231", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 114." - }, - { - "id": "12232", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 115." - }, - { - "id": "12233", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 116." - }, - { - "id": "12234", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 117." - }, - { - "id": "12235", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 118." - }, - { - "id": "12236", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 119." - }, - { - "id": "12237", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 120." - }, - { - "id": "12238", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 121." - }, - { - "id": "12239", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 122." - }, - { - "id": "12240", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 123." - }, - { - "id": "12241", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 124." - }, - { - "id": "12242", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 125." - }, - { - "id": "12243", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 126." - }, - { - "id": "12244", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 127." - }, - { - "id": "12245", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 128." - }, - { - "id": "12246", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 129." - }, - { - "id": "12247", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 130." - }, - { - "id": "12248", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 131." - }, - { - "id": "12249", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 132." - }, - { - "id": "12250", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 133." - }, - { - "id": "12251", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 134." - }, - { - "id": "12252", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 135." - }, - { - "id": "12253", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 136." - }, - { - "id": "12254", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 137." - }, - { - "id": "12255", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 138." - }, - { - "id": "12256", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 139." - }, - { - "id": "12257", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 140." - }, - { - "id": "12258", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 141." - }, - { - "id": "12259", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 142." - }, - { - "id": "12260", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 143." - }, - { - "id": "12261", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 144." - }, - { - "id": "12262", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 145." - }, - { - "id": "12263", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 146." - }, - { - "id": "12264", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 147." - }, - { - "id": "12265", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 148." - }, - { - "id": "12266", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 149." - }, - { - "id": "12267", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 150." - }, - { - "id": "12268", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 151." - }, - { - "id": "12269", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 152." - }, - { - "id": "12270", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 153." - }, - { - "id": "12271", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 154." - }, - { - "id": "12272", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 155." - }, - { - "id": "12273", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 156." - }, - { - "id": "12274", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 157." - }, - { - "id": "12275", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 158." - }, - { - "id": "12276", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 159." - }, - { - "id": "12277", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 160." - }, - { - "id": "12278", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 161." - }, - { - "id": "12279", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 162." - }, - { - "id": "12280", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 163." - }, - { - "id": "12281", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 164." - }, - { - "id": "12282", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 165." - }, - { - "id": "12283", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 166." - }, - { - "id": "12284", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 167." - }, - { - "id": "12285", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 168." - }, - { - "id": "12286", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 169." - }, - { - "id": "12287", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 170." - }, - { - "id": "12288", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 171." - }, - { - "id": "12289", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 172." - }, - { - "id": "12290", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 173." - }, - { - "id": "12291", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 174." - }, - { - "id": "12292", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 175." - }, - { - "id": "12293", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 176." - }, - { - "id": "12294", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 177." - }, - { - "id": "12295", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 178." - }, - { - "id": "12296", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 179." - }, - { - "id": "12297", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 180." - }, - { - "id": "12298", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 181." - }, - { - "id": "12299", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 182." - }, - { - "id": "12300", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 183." - }, - { - "id": "12301", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 184." - }, - { - "id": "12302", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 185." - }, - { - "id": "12303", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 186." - }, - { - "id": "12304", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 187." - }, - { - "id": "12305", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 188." - }, - { - "id": "12306", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 189." - }, - { - "id": "12307", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 190." - }, - { - "id": "12308", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 191." - }, - { - "id": "12309", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 192." - }, - { - "id": "12310", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 193." - }, - { - "id": "12311", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 194." - }, - { - "id": "12312", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 195." - }, - { - "id": "12313", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 196." - }, - { - "id": "12314", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 197." - }, - { - "id": "12315", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 198." - }, - { - "id": "12316", - "chatId": "999", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 199." - }, - { - "id": "12317", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 200." - }, - { - "id": "12318", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 201." - }, - { - "id": "12319", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 202." - }, - { - "id": "12320", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 203." - }, - { - "id": "12321", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 204." - }, - { - "id": "12322", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 205." - }, - { - "id": "12323", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 206." - }, - { - "id": "12324", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 207." - }, - { - "id": "12325", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 208." - }, - { - "id": "12326", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 209." - }, - { - "id": "12327", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 210." - }, - { - "id": "12328", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 211." - }, - { - "id": "12329", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 212." - }, - { - "id": "12330", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 213." - }, - { - "id": "12331", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 214." - }, - { - "id": "12332", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 215." - }, - { - "id": "12333", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 216." - }, - { - "id": "12334", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 217." - }, - { - "id": "12335", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 218." - }, - { - "id": "12336", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 219." - }, - { - "id": "12337", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 220." - }, - { - "id": "12338", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 221." - }, - { - "id": "12339", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 222." - }, - { - "id": "12340", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 223." - }, - { - "id": "12341", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 224." - }, - { - "id": "12342", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 225." - }, - { - "id": "12343", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 226." - }, - { - "id": "12344", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 227." - }, - { - "id": "12345", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 228." - }, - { - "id": "12346", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 229." - }, - { - "id": "12347", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 230." - }, - { - "id": "12348", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 231." - }, - { - "id": "12349", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 232." - }, - { - "id": "12350", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 233." - }, - { - "id": "12351", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 234." - }, - { - "id": "12352", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 235." - }, - { - "id": "12353", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 236." - }, - { - "id": "12354", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 237." - }, - { - "id": "12355", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 238." - }, - { - "id": "12356", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 239." - }, - { - "id": "12357", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 240." - }, - { - "id": "12358", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 241." - }, - { - "id": "12359", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 242." - }, - { - "id": "12360", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 243." - }, - { - "id": "12361", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 244." - }, - { - "id": "12362", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 245." - }, - { - "id": "12363", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 246." - }, - { - "id": "12364", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 247." - }, - { - "id": "12365", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 248." - }, - { - "id": "12366", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 249." - }, - { - "id": "12367", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 250." - }, - { - "id": "12368", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 251." - }, - { - "id": "12369", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 252." - }, - { - "id": "12370", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 253." - }, - { - "id": "12371", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 254." - }, - { - "id": "12372", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 255." - }, - { - "id": "12373", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 256." - }, - { - "id": "12374", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 257." - }, - { - "id": "12375", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 258." - }, - { - "id": "12376", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 259." - }, - { - "id": "12377", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 260." - }, - { - "id": "12378", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 261." - }, - { - "id": "12379", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 262." - }, - { - "id": "12380", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 263." - }, - { - "id": "12381", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 264." - }, - { - "id": "12382", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 265." - }, - { - "id": "12383", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 266." - }, - { - "id": "12384", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 267." - }, - { - "id": "12385", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 268." - }, - { - "id": "12386", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 269." - }, - { - "id": "12387", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 270." - }, - { - "id": "12388", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 271." - }, - { - "id": "12389", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 272." - }, - { - "id": "12390", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 273." - }, - { - "id": "12391", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 274." - }, - { - "id": "12392", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 275." - }, - { - "id": "12393", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 276." - }, - { - "id": "12394", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 277." - }, - { - "id": "12395", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 278." - }, - { - "id": "12396", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 279." - }, - { - "id": "12397", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 280." - }, - { - "id": "12398", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 281." - }, - { - "id": "12399", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 282." - }, - { - "id": "12400", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 283." - }, - { - "id": "12401", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 284." - }, - { - "id": "12402", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 285." - }, - { - "id": "12403", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 286." - }, - { - "id": "12404", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 287." - }, - { - "id": "12405", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 288." - }, - { - "id": "12406", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 289." - }, - { - "id": "12407", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 290." - }, - { - "id": "12408", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 291." - }, - { - "id": "12409", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 292." - }, - { - "id": "12410", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 293." - }, - { - "id": "12411", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 294." - }, - { - "id": "12412", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount: agenda only; no decision yet. Thread 295." - }, - { - "id": "12413", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism renewal discount? Please add this question to the meeting. Thread 296." - }, - { - "id": "12414", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Checklist heading: prism renewal discount. This empty template supplies no policy. Thread 297." - }, - { - "id": "12415", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "prism status digest. renewal is a document tag. discount is a separate training exercise. No answer. Thread 298." - }, - { - "id": "12416", - "chatId": "108", - "senderId": "702", - "timestamp": "2026-10-03T12:00:00.000Z", - "text": "Old prism renewal discount proposal rejected; do not treat this as the agreed decision. Thread 299." - }, - { - "id": "12417", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 0." - }, - { - "id": "12418", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 1." - }, - { - "id": "12419", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 2." - }, - { - "id": "12420", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 3." - }, - { - "id": "12421", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 4." - }, - { - "id": "12422", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 5." - }, - { - "id": "12423", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 6." - }, - { - "id": "12424", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 7." - }, - { - "id": "12425", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 8." - }, - { - "id": "12426", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 9." - }, - { - "id": "12427", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 10." - }, - { - "id": "12428", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 11." - }, - { - "id": "12429", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 12." - }, - { - "id": "12430", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 13." - }, - { - "id": "12431", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 14." - }, - { - "id": "12432", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 15." - }, - { - "id": "12433", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 16." - }, - { - "id": "12434", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 17." - }, - { - "id": "12435", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 18." - }, - { - "id": "12436", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 19." - }, - { - "id": "12437", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 20." - }, - { - "id": "12438", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 21." - }, - { - "id": "12439", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 22." - }, - { - "id": "12440", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 23." - }, - { - "id": "12441", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 24." - }, - { - "id": "12442", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 25." - }, - { - "id": "12443", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 26." - }, - { - "id": "12444", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 27." - }, - { - "id": "12445", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 28." - }, - { - "id": "12446", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 29." - }, - { - "id": "12447", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 30." - }, - { - "id": "12448", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 31." - }, - { - "id": "12449", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 32." - }, - { - "id": "12450", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 33." - }, - { - "id": "12451", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 34." - }, - { - "id": "12452", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 35." - }, - { - "id": "12453", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 36." - }, - { - "id": "12454", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 37." - }, - { - "id": "12455", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 38." - }, - { - "id": "12456", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 39." - }, - { - "id": "12457", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 40." - }, - { - "id": "12458", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 41." - }, - { - "id": "12459", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 42." - }, - { - "id": "12460", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 43." - }, - { - "id": "12461", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 44." - }, - { - "id": "12462", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 45." - }, - { - "id": "12463", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 46." - }, - { - "id": "12464", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 47." - }, - { - "id": "12465", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 48." - }, - { - "id": "12466", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 49." - }, - { - "id": "12467", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 50." - }, - { - "id": "12468", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 51." - }, - { - "id": "12469", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 52." - }, - { - "id": "12470", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 53." - }, - { - "id": "12471", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 54." - }, - { - "id": "12472", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 55." - }, - { - "id": "12473", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 56." - }, - { - "id": "12474", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 57." - }, - { - "id": "12475", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 58." - }, - { - "id": "12476", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 59." - }, - { - "id": "12477", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 60." - }, - { - "id": "12478", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 61." - }, - { - "id": "12479", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 62." - }, - { - "id": "12480", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 63." - }, - { - "id": "12481", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 64." - }, - { - "id": "12482", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 65." - }, - { - "id": "12483", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 66." - }, - { - "id": "12484", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 67." - }, - { - "id": "12485", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 68." - }, - { - "id": "12486", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 69." - }, - { - "id": "12487", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 70." - }, - { - "id": "12488", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 71." - }, - { - "id": "12489", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 72." - }, - { - "id": "12490", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 73." - }, - { - "id": "12491", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 74." - }, - { - "id": "12492", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 75." - }, - { - "id": "12493", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 76." - }, - { - "id": "12494", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 77." - }, - { - "id": "12495", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 78." - }, - { - "id": "12496", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 79." - }, - { - "id": "12497", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 80." - }, - { - "id": "12498", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 81." - }, - { - "id": "12499", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 82." - }, - { - "id": "12500", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 83." - }, - { - "id": "12501", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 84." - }, - { - "id": "12502", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 85." - }, - { - "id": "12503", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 86." - }, - { - "id": "12504", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 87." - }, - { - "id": "12505", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 88." - }, - { - "id": "12506", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 89." - }, - { - "id": "12507", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 90." - }, - { - "id": "12508", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 91." - }, - { - "id": "12509", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 92." - }, - { - "id": "12510", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 93." - }, - { - "id": "12511", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 94." - }, - { - "id": "12512", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 95." - }, - { - "id": "12513", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 96." - }, - { - "id": "12514", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 97." - }, - { - "id": "12515", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 98." - }, - { - "id": "12516", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 99." - }, - { - "id": "12517", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 100." - }, - { - "id": "12518", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 101." - }, - { - "id": "12519", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 102." - }, - { - "id": "12520", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 103." - }, - { - "id": "12521", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 104." - }, - { - "id": "12522", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 105." - }, - { - "id": "12523", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 106." - }, - { - "id": "12524", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 107." - }, - { - "id": "12525", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 108." - }, - { - "id": "12526", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 109." - }, - { - "id": "12527", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 110." - }, - { - "id": "12528", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 111." - }, - { - "id": "12529", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 112." - }, - { - "id": "12530", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 113." - }, - { - "id": "12531", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 114." - }, - { - "id": "12532", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 115." - }, - { - "id": "12533", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 116." - }, - { - "id": "12534", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 117." - }, - { - "id": "12535", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 118." - }, - { - "id": "12536", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 119." - }, - { - "id": "12537", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 120." - }, - { - "id": "12538", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 121." - }, - { - "id": "12539", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 122." - }, - { - "id": "12540", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 123." - }, - { - "id": "12541", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 124." - }, - { - "id": "12542", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 125." - }, - { - "id": "12543", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 126." - }, - { - "id": "12544", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 127." - }, - { - "id": "12545", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 128." - }, - { - "id": "12546", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 129." - }, - { - "id": "12547", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 130." - }, - { - "id": "12548", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 131." - }, - { - "id": "12549", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 132." - }, - { - "id": "12550", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 133." - }, - { - "id": "12551", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 134." - }, - { - "id": "12552", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 135." - }, - { - "id": "12553", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 136." - }, - { - "id": "12554", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 137." - }, - { - "id": "12555", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 138." - }, - { - "id": "12556", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 139." - }, - { - "id": "12557", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 140." - }, - { - "id": "12558", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 141." - }, - { - "id": "12559", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 142." - }, - { - "id": "12560", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 143." - }, - { - "id": "12561", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 144." - }, - { - "id": "12562", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 145." - }, - { - "id": "12563", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 146." - }, - { - "id": "12564", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 147." - }, - { - "id": "12565", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 148." - }, - { - "id": "12566", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 149." - }, - { - "id": "12567", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 150." - }, - { - "id": "12568", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 151." - }, - { - "id": "12569", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 152." - }, - { - "id": "12570", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 153." - }, - { - "id": "12571", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 154." - }, - { - "id": "12572", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 155." - }, - { - "id": "12573", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 156." - }, - { - "id": "12574", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 157." - }, - { - "id": "12575", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 158." - }, - { - "id": "12576", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 159." - }, - { - "id": "12577", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 160." - }, - { - "id": "12578", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 161." - }, - { - "id": "12579", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 162." - }, - { - "id": "12580", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 163." - }, - { - "id": "12581", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 164." - }, - { - "id": "12582", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 165." - }, - { - "id": "12583", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 166." - }, - { - "id": "12584", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 167." - }, - { - "id": "12585", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 168." - }, - { - "id": "12586", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 169." - }, - { - "id": "12587", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 170." - }, - { - "id": "12588", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 171." - }, - { - "id": "12589", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 172." - }, - { - "id": "12590", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 173." - }, - { - "id": "12591", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 174." - }, - { - "id": "12592", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 175." - }, - { - "id": "12593", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 176." - }, - { - "id": "12594", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 177." - }, - { - "id": "12595", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 178." - }, - { - "id": "12596", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 179." - }, - { - "id": "12597", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 180." - }, - { - "id": "12598", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 181." - }, - { - "id": "12599", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 182." - }, - { - "id": "12600", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 183." - }, - { - "id": "12601", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 184." - }, - { - "id": "12602", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 185." - }, - { - "id": "12603", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 186." - }, - { - "id": "12604", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 187." - }, - { - "id": "12605", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 188." - }, - { - "id": "12606", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 189." - }, - { - "id": "12607", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 190." - }, - { - "id": "12608", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 191." - }, - { - "id": "12609", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 192." - }, - { - "id": "12610", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 193." - }, - { - "id": "12611", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 194." - }, - { - "id": "12612", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 195." - }, - { - "id": "12613", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 196." - }, - { - "id": "12614", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 197." - }, - { - "id": "12615", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 198." - }, - { - "id": "12616", - "chatId": "999", - "senderId": "703", - "timestamp": "2026-10-04T12:00:00.000Z", - "text": "Unrelated lunch timetable and office stationery reminder 199." - } - ], - "queries": [ - { - "id": "aurora-words", - "split": "dev", - "category": "words", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora rollout rollback", - "legacy": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-typo", - "split": "dev", - "category": "typo", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora rolluot rollback", - "legacy": "aurora rolluot rollback", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-beginnings", - "split": "dev", - "category": "beginnings", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora roll rollback", - "legacy": "aurora roll rollback", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-phrase", - "split": "dev", - "category": "phrase", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "\"aurora rollout rollback\"", - "legacy": "\"aurora rollout rollback\"", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-from", - "split": "dev", - "category": "from", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora rollout from:700", - "legacy": "aurora rollout from:700", - "relevant": { - "10001": 2 - } - }, - { - "id": "aurora-chat", - "split": "dev", - "category": "chat", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora rollout chat:101", - "legacy": "aurora rollout chat:101", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-date", - "split": "dev", - "category": "date", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora rollout date:2026-10-02", - "legacy": "aurora rollout after:2026-10-02 before:2026-10-03", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-and", - "split": "dev", - "category": "and", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora AND rollout AND rollback", - "legacy": "aurora rollout rollback", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-or", - "split": "dev", - "category": "or", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora rollout OR rollback", - "legacy": "aurora rollout OR rollback", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-paraphrase", - "split": "dev", - "category": "paraphrase", - "intent": "Aurora rollout rollback decision: disable the canary and restore build 17.", - "lucene": "aurora deployment reversal", - "legacy": "aurora deployment reversal", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-absent", - "split": "dev", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "aurora quantum unicorn", - "legacy": "aurora quantum unicorn", - "relevant": {} - }, - { - "id": "aurora-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "aurora rollout rollback purple", - "legacy": "aurora rollout rollback purple", - "relevant": {} - }, - { - "id": "cedar-words", - "split": "dev", - "category": "words", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar backup retention", - "legacy": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-typo", - "split": "dev", - "category": "typo", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar bakcup retention", - "legacy": "cedar bakcup retention", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-beginnings", - "split": "dev", - "category": "beginnings", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar back retention", - "legacy": "cedar back retention", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-phrase", - "split": "dev", - "category": "phrase", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "\"cedar backup retention\"", - "legacy": "\"cedar backup retention\"", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-from", - "split": "dev", - "category": "from", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar backup from:700", - "legacy": "cedar backup from:700", - "relevant": { - "10303": 2 - } - }, - { - "id": "cedar-chat", - "split": "dev", - "category": "chat", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar backup chat:102", - "legacy": "cedar backup chat:102", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-date", - "split": "dev", - "category": "date", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar backup date:2026-10-02", - "legacy": "cedar backup after:2026-10-02 before:2026-10-03", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-and", - "split": "dev", - "category": "and", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar AND backup AND retention", - "legacy": "cedar backup retention", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-or", - "split": "dev", - "category": "or", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar backup OR retention", - "legacy": "cedar backup OR retention", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-paraphrase", - "split": "dev", - "category": "paraphrase", - "intent": "Cedar backup retention decision: keep daily snapshots for 35 days.", - "lucene": "cedar snapshot lifetime", - "legacy": "cedar snapshot lifetime", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-absent", - "split": "dev", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "cedar quantum unicorn", - "legacy": "cedar quantum unicorn", - "relevant": {} - }, - { - "id": "cedar-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "cedar backup retention purple", - "legacy": "cedar backup retention purple", - "relevant": {} - }, - { - "id": "север-words", - "split": "dev", - "category": "words", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север сертификат обновление", - "legacy": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-typo", - "split": "dev", - "category": "typo", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север сертифкат обновление", - "legacy": "север сертифкат обновление", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-beginnings", - "split": "dev", - "category": "beginnings", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север серти обновление", - "legacy": "север серти обновление", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-phrase", - "split": "dev", - "category": "phrase", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "\"север сертификат обновление\"", - "legacy": "\"север сертификат обновление\"", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-from", - "split": "dev", - "category": "from", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север сертификат from:700", - "legacy": "север сертификат from:700", - "relevant": { - "10605": 2 - } - }, - { - "id": "север-chat", - "split": "dev", - "category": "chat", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север сертификат chat:103", - "legacy": "север сертификат chat:103", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-date", - "split": "dev", - "category": "date", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север сертификат date:2026-10-02", - "legacy": "север сертификат after:2026-10-02 before:2026-10-03", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-and", - "split": "dev", - "category": "and", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север AND сертификат AND обновление", - "legacy": "север сертификат обновление", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-or", - "split": "dev", - "category": "or", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север сертификат OR обновление", - "legacy": "север сертификат OR обновление", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-paraphrase", - "split": "dev", - "category": "paraphrase", - "intent": "Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.", - "lucene": "север продление TLS", - "legacy": "север продление TLS", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-absent", - "split": "dev", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "север quantum unicorn", - "legacy": "север quantum unicorn", - "relevant": {} - }, - { - "id": "север-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "север сертификат обновление purple", - "legacy": "север сертификат обновление purple", - "relevant": {} - }, - { - "id": "lumen-words", - "split": "dev", - "category": "words", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen invoice refund", - "legacy": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-typo", - "split": "dev", - "category": "typo", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen invocie refund", - "legacy": "lumen invocie refund", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-beginnings", - "split": "dev", - "category": "beginnings", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen invo refund", - "legacy": "lumen invo refund", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-phrase", - "split": "dev", - "category": "phrase", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "\"lumen invoice refund\"", - "legacy": "\"lumen invoice refund\"", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-from", - "split": "dev", - "category": "from", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen invoice from:700", - "legacy": "lumen invoice from:700", - "relevant": { - "10907": 2 - } - }, - { - "id": "lumen-chat", - "split": "dev", - "category": "chat", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen invoice chat:104", - "legacy": "lumen invoice chat:104", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-date", - "split": "dev", - "category": "date", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen invoice date:2026-10-02", - "legacy": "lumen invoice after:2026-10-02 before:2026-10-03", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-and", - "split": "dev", - "category": "and", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen AND invoice AND refund", - "legacy": "lumen invoice refund", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-or", - "split": "dev", - "category": "or", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen invoice OR refund", - "legacy": "lumen invoice OR refund", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-paraphrase", - "split": "dev", - "category": "paraphrase", - "intent": "Lumen invoice refund approved: return 480 credits to the original payer.", - "lucene": "lumen payment reimbursement", - "legacy": "lumen payment reimbursement", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-absent", - "split": "dev", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "lumen quantum unicorn", - "legacy": "lumen quantum unicorn", - "relevant": {} - }, - { - "id": "lumen-near-miss", - "split": "dev", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "lumen invoice refund purple", - "legacy": "lumen invoice refund purple", - "relevant": {} - }, - { - "id": "harbor-words", - "split": "test", - "category": "words", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor release freeze", - "legacy": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-typo", - "split": "test", - "category": "typo", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor relesae freeze", - "legacy": "harbor relesae freeze", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-beginnings", - "split": "test", - "category": "beginnings", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor rele freeze", - "legacy": "harbor rele freeze", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-phrase", - "split": "test", - "category": "phrase", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "\"harbor release freeze\"", - "legacy": "\"harbor release freeze\"", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-from", - "split": "test", - "category": "from", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor release from:700", - "legacy": "harbor release from:700", - "relevant": { - "11209": 2 - } - }, - { - "id": "harbor-chat", - "split": "test", - "category": "chat", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor release chat:105", - "legacy": "harbor release chat:105", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-date", - "split": "test", - "category": "date", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor release date:2026-10-02", - "legacy": "harbor release after:2026-10-02 before:2026-10-03", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-and", - "split": "test", - "category": "and", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor AND release AND freeze", - "legacy": "harbor release freeze", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-or", - "split": "test", - "category": "or", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor release OR freeze", - "legacy": "harbor release OR freeze", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-paraphrase", - "split": "test", - "category": "paraphrase", - "intent": "Harbor release freeze decision: stop deployments until the database check passes.", - "lucene": "harbor shipping pause", - "legacy": "harbor shipping pause", - "relevant": { - "11209": 2, - "11210": 1 - } - }, - { - "id": "harbor-absent", - "split": "test", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "harbor quantum unicorn", - "legacy": "harbor quantum unicorn", - "relevant": {} - }, - { - "id": "harbor-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "harbor release freeze purple", - "legacy": "harbor release freeze purple", - "relevant": {} - }, - { - "id": "maple-words", - "split": "test", - "category": "words", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple archive restore", - "legacy": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-typo", - "split": "test", - "category": "typo", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple arhcive restore", - "legacy": "maple arhcive restore", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-beginnings", - "split": "test", - "category": "beginnings", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple arch restore", - "legacy": "maple arch restore", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-phrase", - "split": "test", - "category": "phrase", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "\"maple archive restore\"", - "legacy": "\"maple archive restore\"", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-from", - "split": "test", - "category": "from", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple archive from:700", - "legacy": "maple archive from:700", - "relevant": { - "11511": 2 - } - }, - { - "id": "maple-chat", - "split": "test", - "category": "chat", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple archive chat:106", - "legacy": "maple archive chat:106", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-date", - "split": "test", - "category": "date", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple archive date:2026-10-02", - "legacy": "maple archive after:2026-10-02 before:2026-10-03", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-and", - "split": "test", - "category": "and", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple AND archive AND restore", - "legacy": "maple archive restore", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-or", - "split": "test", - "category": "or", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple archive OR restore", - "legacy": "maple archive OR restore", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-paraphrase", - "split": "test", - "category": "paraphrase", - "intent": "Maple archive restore decision: recover the March snapshot into a separate workspace.", - "lucene": "maple recover saved data", - "legacy": "maple recover saved data", - "relevant": { - "11511": 2, - "11512": 1 - } - }, - { - "id": "maple-absent", - "split": "test", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "maple quantum unicorn", - "legacy": "maple quantum unicorn", - "relevant": {} - }, - { - "id": "maple-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "maple archive restore purple", - "legacy": "maple archive restore purple", - "relevant": {} - }, - { - "id": "вектор-words", - "split": "test", - "category": "words", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор миграция отмена", - "legacy": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-typo", - "split": "test", - "category": "typo", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор миграця отмена", - "legacy": "вектор миграця отмена", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-beginnings", - "split": "test", - "category": "beginnings", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор мигр отмена", - "legacy": "вектор мигр отмена", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-phrase", - "split": "test", - "category": "phrase", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "\"вектор миграция отмена\"", - "legacy": "\"вектор миграция отмена\"", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-from", - "split": "test", - "category": "from", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор миграция from:700", - "legacy": "вектор миграция from:700", - "relevant": { - "11813": 2 - } - }, - { - "id": "вектор-chat", - "split": "test", - "category": "chat", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор миграция chat:107", - "legacy": "вектор миграция chat:107", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-date", - "split": "test", - "category": "date", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор миграция date:2026-10-02", - "legacy": "вектор миграция after:2026-10-02 before:2026-10-03", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-and", - "split": "test", - "category": "and", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор AND миграция AND отмена", - "legacy": "вектор миграция отмена", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-or", - "split": "test", - "category": "or", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор миграция OR отмена", - "legacy": "вектор миграция OR отмена", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-paraphrase", - "split": "test", - "category": "paraphrase", - "intent": "Вектор миграция отмена: отключить задание и вернуть схему версии 8.", - "lucene": "вектор возврат прежней схемы", - "legacy": "вектор возврат прежней схемы", - "relevant": { - "11813": 2, - "11814": 1 - } - }, - { - "id": "вектор-absent", - "split": "test", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "вектор quantum unicorn", - "legacy": "вектор quantum unicorn", - "relevant": {} - }, - { - "id": "вектор-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "вектор миграция отмена purple", - "legacy": "вектор миграция отмена purple", - "relevant": {} - }, - { - "id": "prism-words", - "split": "test", - "category": "words", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism renewal discount", - "legacy": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-typo", - "split": "test", - "category": "typo", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism renweal discount", - "legacy": "prism renweal discount", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-beginnings", - "split": "test", - "category": "beginnings", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism rene discount", - "legacy": "prism rene discount", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-phrase", - "split": "test", - "category": "phrase", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "\"prism renewal discount\"", - "legacy": "\"prism renewal discount\"", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-from", - "split": "test", - "category": "from", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism renewal from:700", - "legacy": "prism renewal from:700", - "relevant": { - "12115": 2 - } - }, - { - "id": "prism-chat", - "split": "test", - "category": "chat", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism renewal chat:108", - "legacy": "prism renewal chat:108", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-date", - "split": "test", - "category": "date", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism renewal date:2026-10-02", - "legacy": "prism renewal after:2026-10-02 before:2026-10-03", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-and", - "split": "test", - "category": "and", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism AND renewal AND discount", - "legacy": "prism renewal discount", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-or", - "split": "test", - "category": "or", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism renewal OR discount", - "legacy": "prism renewal OR discount", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-paraphrase", - "split": "test", - "category": "paraphrase", - "intent": "Prism renewal discount approved: apply 15 percent to the annual contract.", - "lucene": "prism subscription price reduction", - "legacy": "prism subscription price reduction", - "relevant": { - "12115": 2, - "12116": 1 - } - }, - { - "id": "prism-absent", - "split": "test", - "category": "no-answer-absent", - "intent": "No message answers the requested combination.", - "lucene": "prism quantum unicorn", - "legacy": "prism quantum unicorn", - "relevant": {} - }, - { - "id": "prism-near-miss", - "split": "test", - "category": "no-answer-near-miss", - "intent": "No message answers the requested combination.", - "lucene": "prism renewal discount purple", - "legacy": "prism renewal discount purple", - "relevant": {} - } - ] -} diff --git a/bench/message-search-quality/crossencoder-dev.json b/bench/message-search-quality/crossencoder-dev.json deleted file mode 100644 index d452cc3b5..000000000 --- a/bench/message-search-quality/crossencoder-dev.json +++ /dev/null @@ -1,30187 +0,0 @@ -{ - "crossEncoderImplementationSha256": "2d044b35dc1f4ac769127e1b86303b5b4d20f7970eb2ea9a47df729153e96aa0", - "rerankerManifestSha256": "8efb505319d19cb42772dc02438cd99c68cfd4771bcba8890aac3ec81b1538d3", - "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", - "questionsSha256": "390ff1475964d982a06afaf9e2f6f840f5b8304fcffe58ebedd5a0ea8d54478e", - "runnerSha256": "bea007961d0dc1d237824c274029f81dce28c4840e8df88c9b2b282d9bf07fd5", - "productionSha256": { - "services/messages-combined.js": "d10aeb5f7049f615503d59c37df109bcd284f53b5cfceaac0151f2d7e2ca60e9", - "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", - "embeddings/embed.js": "cacf0bec8b98aeda4ed135642d593c4d5b50800867bc8e95de0ad77c1e002e3f", - "embeddings/models.js": "8debfaa01509728ba206702f27e6d8251b8968a03375089b6ca1af9d0eb53e94", - "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964" - }, - "node": "v24.19.0", - "cpu": "AMD Ryzen AI 9 HX 470 w/ Radeon 890M", - "storePath": "/tmp/semantic-message-quality-rDgMNR/quality.db", - "model": { - "id": "cross-encoder/mmarco-mMiniLMv2-L12-H384-v1", - "revision": "1427fd652930e4ba29e8149678df786c240d8825", - "licence": "Apache-2.0", - "files": [ - { - "name": "onnx/model_quint8_avx2.onnx", - "sha256": "6c2513767fb63d008a4377bef7a7a3555433d9436342bb53e35a3a72ffc52d4b" - }, - { - "name": "tokenizer.json", - "sha256": "62c24cdc13d4c9952d63718d6c9fa4c287974249e16b7ade6d5a85e7bbb75626" - }, - { - "name": "tokenizer_config.json", - "sha256": "e7fbfbfa6347b4e414c1cee50d142e2c2f9a895dad68b068ae83a8b564c3837e" - }, - { - "name": "config.json", - "sha256": "cc2cfe51aa3fd759d21d21acf5dfd6994aa67a3c9210636d22e143699d336c77" - } - ], - "threads": 8 - }, - "kind": "cross-encoder raw logit reranking", - "candidateSummary": [ - { - "depth": 50, - "candidateRecall": 0.4375, - "byCategory": { - "words": 0.25, - "typo": 0.25, - "beginnings": 0.25, - "phrase": 0.125, - "from": 1, - "chat": 1, - "date": 1, - "and": 0.25, - "or": 0.25, - "paraphrase": 0 - } - }, - { - "depth": 100, - "candidateRecall": 0.75, - "byCategory": { - "words": 0.75, - "typo": 0.75, - "beginnings": 0.75, - "phrase": 0.75, - "from": 1, - "chat": 1, - "date": 1, - "and": 0.75, - "or": 0.75, - "paraphrase": 0 - } - }, - { - "depth": 300, - "candidateRecall": 0.9, - "byCategory": { - "words": 1, - "typo": 1, - "beginnings": 1, - "phrase": 1, - "from": 1, - "chat": 1, - "date": 1, - "and": 1, - "or": 1, - "paraphrase": 0 - } - }, - { - "depth": 500, - "candidateRecall": 0.9, - "byCategory": { - "words": 1, - "typo": 1, - "beginnings": 1, - "phrase": 1, - "from": 1, - "chat": 1, - "date": 1, - "and": 1, - "or": 1, - "paraphrase": 0 - } - } - ], - "selectedOnDev": { - "depth": 300, - "questionScoreFloor": 2 - }, - "keywordSummary": { - "lexical": { - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0 - }, - "semantic": { - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.525, - "mrr": 0.553125, - "ndcgAt10": 0.4769238266733387, - "noAnswerFalseHits": 0 - } - }, - "questionSummary": { - "direct": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - "anchoredLexical": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 40 - }, - "sweep": [ - { - "floor": null, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.6875, - "mrr": 0.875, - "ndcgAt10": 0.7735661101175583, - "noAnswerFalseHits": 40 - } - }, - { - "floor": -5, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.6875, - "mrr": 0.875, - "ndcgAt10": 0.7735661101175583, - "noAnswerFalseHits": 40 - } - }, - { - "floor": -2, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.625, - "mrr": 0.875, - "ndcgAt10": 0.7627057761880933, - "noAnswerFalseHits": 13 - } - }, - { - "floor": -1, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.625, - "mrr": 0.875, - "ndcgAt10": 0.7627057761880933, - "noAnswerFalseHits": 6 - } - }, - { - "floor": 0, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.625, - "mrr": 0.875, - "ndcgAt10": 0.7627057761880933, - "noAnswerFalseHits": 3 - } - }, - { - "floor": 1, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.875, - "ndcgAt10": 0.7409851083291633, - "noAnswerFalseHits": 1 - } - }, - { - "floor": 2, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.875, - "ndcgAt10": 0.7409851083291633, - "noAnswerFalseHits": 0 - } - }, - { - "floor": 3, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.875, - "ndcgAt10": 0.7409851083291633, - "noAnswerFalseHits": 0 - } - }, - { - "floor": 5, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.875, - "ndcgAt10": 0.7409851083291633, - "noAnswerFalseHits": 0 - } - }, - { - "floor": 8, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.25, - "mrr": 0.5, - "ndcgAt10": 0.41311732856427996, - "noAnswerFalseHits": 0 - } - } - ] - }, - "devQualified": true, - "questionRetrieval": "manual topic anchors for diagnostic reranking; full-question direct retrieval is measured separately", - "heldOut": { - "keyword": { - "lexical": { - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.275, - "mrr": 0.35, - "ndcgAt10": 0.226064801430716, - "noAnswerFalseHits": 0 - }, - "semantic": { - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.4375, - "mrr": 0.4875, - "ndcgAt10": 0.40487848841280566, - "noAnswerFalseHits": 0 - }, - "rows": [ - { - "id": "harbor-words", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11210", - "score": 9.45776081085205, - "lexicalRank": 121 - }, - { - "id": "11209", - "score": 9.25239086151123, - "lexicalRank": 60 - }, - { - "id": "11459", - "score": 8.542092323303223, - "lexicalRank": 252 - }, - { - "id": "11319", - "score": 8.532307624816895, - "lexicalRank": 280 - }, - { - "id": "11449", - "score": 8.469769477844238, - "lexicalRank": 254 - }, - { - "id": "11409", - "score": 8.397802352905273, - "lexicalRank": 262 - }, - { - "id": "11439", - "score": 8.30823802947998, - "lexicalRank": 256 - }, - { - "id": "11344", - "score": 8.291367530822754, - "lexicalRank": 275 - }, - { - "id": "11509", - "score": 8.277020454406738, - "lexicalRank": 242 - }, - { - "id": "11419", - "score": 8.27120304107666, - "lexicalRank": 260 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-typo", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "score": 8.698358535766602, - "lexicalRank": 60 - }, - { - "id": "11210", - "score": 8.513386726379395, - "lexicalRank": 121 - }, - { - "id": "11481", - "score": 7.69588041305542, - "lexicalRank": 5 - }, - { - "id": "11446", - "score": 7.607393741607666, - "lexicalRank": 12 - }, - { - "id": "11346", - "score": 7.577287673950195, - "lexicalRank": 32 - }, - { - "id": "11256", - "score": 7.526706218719482, - "lexicalRank": 50 - }, - { - "id": "11391", - "score": 7.5081915855407715, - "lexicalRank": 23 - }, - { - "id": "11451", - "score": 7.493976593017578, - "lexicalRank": 11 - }, - { - "id": "11381", - "score": 7.487243175506592, - "lexicalRank": 25 - }, - { - "id": "11501", - "score": 7.483785152435303, - "lexicalRank": 1 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-beginnings", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11209", - "score": 9.069320678710938, - "lexicalRank": 60 - }, - { - "id": "11210", - "score": 8.7719144821167, - "lexicalRank": 121 - }, - { - "id": "11301", - "score": 7.857519626617432, - "lexicalRank": 41 - }, - { - "id": "11416", - "score": 7.742708683013916, - "lexicalRank": 18 - }, - { - "id": "11366", - "score": 7.652957916259766, - "lexicalRank": 28 - }, - { - "id": "11211", - "score": 7.649582862854004, - "lexicalRank": 59 - }, - { - "id": "11291", - "score": 7.621363639831543, - "lexicalRank": 43 - }, - { - "id": "11321", - "score": 7.560271263122559, - "lexicalRank": 37 - }, - { - "id": "11346", - "score": 7.559664249420166, - "lexicalRank": 32 - }, - { - "id": "11431", - "score": 7.4946208000183105, - "lexicalRank": 15 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-phrase", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "score": 2.7623348535297314 - }, - { - "id": "11501", - "score": 2.7623348535297314 - }, - { - "id": "11496", - "score": 2.7623348535297314 - }, - { - "id": "11491", - "score": 2.7623348535297314 - }, - { - "id": "11486", - "score": 2.7623348535297314 - }, - { - "id": "11481", - "score": 2.7623348535297314 - }, - { - "id": "11476", - "score": 2.7623348535297314 - }, - { - "id": "11471", - "score": 2.7623348535297314 - }, - { - "id": "11466", - "score": 2.7623348535297314 - }, - { - "id": "11461", - "score": 2.7623348535297314 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-from", - "relevant": { - "11209": 2 - }, - "hits": [ - { - "id": "11209", - "score": -2.535362720489502, - "lexicalRank": 0 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - } - }, - { - "id": "harbor-chat", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11454", - "score": -0.74288409948349, - "lexicalRank": 93 - }, - { - "id": "11494", - "score": -0.8699719309806824, - "lexicalRank": 85 - }, - { - "id": "11414", - "score": -0.9635332226753235, - "lexicalRank": 101 - }, - { - "id": "11424", - "score": -1.1062660217285156, - "lexicalRank": 99 - }, - { - "id": "11444", - "score": -1.1148967742919922, - "lexicalRank": 95 - }, - { - "id": "11434", - "score": -1.1507675647735596, - "lexicalRank": 97 - }, - { - "id": "11489", - "score": -1.187790036201477, - "lexicalRank": 86 - }, - { - "id": "11504", - "score": -1.1919387578964233, - "lexicalRank": 83 - }, - { - "id": "11479", - "score": -1.2202425003051758, - "lexicalRank": 88 - }, - { - "id": "11429", - "score": -1.2790064811706543, - "lexicalRank": 98 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-date", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11210", - "score": -3.488126754760742, - "lexicalRank": 1 - }, - { - "id": "11209", - "score": -3.87473201751709, - "lexicalRank": 0 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - } - }, - { - "id": "harbor-and", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "score": 7.489838832938161 - }, - { - "id": "11501", - "score": 7.489838832938161 - }, - { - "id": "11496", - "score": 7.489838832938161 - }, - { - "id": "11491", - "score": 7.489838832938161 - }, - { - "id": "11486", - "score": 7.489838832938161 - }, - { - "id": "11481", - "score": 7.489838832938161 - }, - { - "id": "11476", - "score": 7.489838832938161 - }, - { - "id": "11471", - "score": 7.489838832938161 - }, - { - "id": "11466", - "score": 7.489838832938161 - }, - { - "id": "11461", - "score": 7.489838832938161 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-or", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [ - { - "id": "11506", - "score": 7.489838832938161 - }, - { - "id": "11501", - "score": 7.489838832938161 - }, - { - "id": "11496", - "score": 7.489838832938161 - }, - { - "id": "11491", - "score": 7.489838832938161 - }, - { - "id": "11486", - "score": 7.489838832938161 - }, - { - "id": "11481", - "score": 7.489838832938161 - }, - { - "id": "11476", - "score": 7.489838832938161 - }, - { - "id": "11471", - "score": 7.489838832938161 - }, - { - "id": "11466", - "score": 7.489838832938161 - }, - { - "id": "11461", - "score": 7.489838832938161 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-paraphrase", - "relevant": { - "11209": 2, - "11210": 1 - }, - "hits": [], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "harbor-absent", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "harbor-near-miss", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "maple-words", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11593", - "score": 7.788802146911621, - "lexicalRank": 43 - }, - { - "id": "11656", - "score": 7.6396074295043945, - "lexicalRank": 273 - }, - { - "id": "11671", - "score": 7.6235737800598145, - "lexicalRank": 270 - }, - { - "id": "11751", - "score": 7.563724517822266, - "lexicalRank": 254 - }, - { - "id": "11553", - "score": 7.5539326667785645, - "lexicalRank": 51 - }, - { - "id": "11768", - "score": 7.544630527496338, - "lexicalRank": 8 - }, - { - "id": "11716", - "score": 7.518007278442383, - "lexicalRank": 261 - }, - { - "id": "11756", - "score": 7.508123874664307, - "lexicalRank": 253 - }, - { - "id": "11701", - "score": 7.493647575378418, - "lexicalRank": 264 - }, - { - "id": "11653", - "score": 7.427809715270996, - "lexicalRank": 31 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-typo", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11511", - "score": 4.387134075164795, - "lexicalRank": 121 - }, - { - "id": "11593", - "score": 4.083454608917236, - "lexicalRank": 43 - }, - { - "id": "11693", - "score": 3.98818302154541, - "lexicalRank": 23 - }, - { - "id": "11708", - "score": 3.9621403217315674, - "lexicalRank": 20 - }, - { - "id": "11788", - "score": 3.935478687286377, - "lexicalRank": 4 - }, - { - "id": "11738", - "score": 3.922764539718628, - "lexicalRank": 14 - }, - { - "id": "11803", - "score": 3.8401293754577637, - "lexicalRank": 1 - }, - { - "id": "11763", - "score": 3.818089485168457, - "lexicalRank": 9 - }, - { - "id": "11798", - "score": 3.810894250869751, - "lexicalRank": 2 - }, - { - "id": "11768", - "score": 3.8016297817230225, - "lexicalRank": 8 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-beginnings", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11512", - "score": 4.051966190338135, - "lexicalRank": 120 - }, - { - "id": "11633", - "score": 3.726194381713867, - "lexicalRank": 35 - }, - { - "id": "11717", - "score": 3.639094829559326, - "lexicalRank": 201 - }, - { - "id": "11511", - "score": 3.412858009338379, - "lexicalRank": 121 - }, - { - "id": "11618", - "score": 3.4080142974853516, - "lexicalRank": 38 - }, - { - "id": "11738", - "score": 3.2612063884735107, - "lexicalRank": 14 - }, - { - "id": "11538", - "score": 3.1791515350341797, - "lexicalRank": 54 - }, - { - "id": "11767", - "score": 3.1666500568389893, - "lexicalRank": 191 - }, - { - "id": "11773", - "score": 3.1325058937072754, - "lexicalRank": 7 - }, - { - "id": "11798", - "score": 3.0707685947418213, - "lexicalRank": 2 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.631251450669265, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-phrase", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "score": 2.7623348535297314 - }, - { - "id": "11803", - "score": 2.7623348535297314 - }, - { - "id": "11798", - "score": 2.7623348535297314 - }, - { - "id": "11793", - "score": 2.7623348535297314 - }, - { - "id": "11788", - "score": 2.7623348535297314 - }, - { - "id": "11783", - "score": 2.7623348535297314 - }, - { - "id": "11778", - "score": 2.7623348535297314 - }, - { - "id": "11773", - "score": 2.7623348535297314 - }, - { - "id": "11768", - "score": 2.7623348535297314 - }, - { - "id": "11763", - "score": 2.7623348535297314 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-from", - "relevant": { - "11511": 2 - }, - "hits": [ - { - "id": "11511", - "score": -3.522339105606079, - "lexicalRank": 0 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - } - }, - { - "id": "maple-chat", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11736", - "score": -1.9755055904388428, - "lexicalRank": 97 - }, - { - "id": "11716", - "score": -2.040720224380493, - "lexicalRank": 101 - }, - { - "id": "11788", - "score": -2.0611703395843506, - "lexicalRank": 4 - }, - { - "id": "11793", - "score": -2.237440347671509, - "lexicalRank": 3 - }, - { - "id": "11721", - "score": -2.247375249862671, - "lexicalRank": 100 - }, - { - "id": "11761", - "score": -2.2613048553466797, - "lexicalRank": 92 - }, - { - "id": "11726", - "score": -2.2749173641204834, - "lexicalRank": 99 - }, - { - "id": "11801", - "score": -2.2842514514923096, - "lexicalRank": 84 - }, - { - "id": "11771", - "score": -2.3484065532684326, - "lexicalRank": 90 - }, - { - "id": "11746", - "score": -2.3657286167144775, - "lexicalRank": 95 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-date", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11511", - "score": -3.7387211322784424, - "lexicalRank": 1 - }, - { - "id": "11512", - "score": -3.991640329360962, - "lexicalRank": 0 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - } - }, - { - "id": "maple-and", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "score": 7.48582930091083 - }, - { - "id": "11803", - "score": 7.48582930091083 - }, - { - "id": "11798", - "score": 7.48582930091083 - }, - { - "id": "11793", - "score": 7.48582930091083 - }, - { - "id": "11788", - "score": 7.48582930091083 - }, - { - "id": "11783", - "score": 7.48582930091083 - }, - { - "id": "11778", - "score": 7.48582930091083 - }, - { - "id": "11773", - "score": 7.48582930091083 - }, - { - "id": "11768", - "score": 7.48582930091083 - }, - { - "id": "11763", - "score": 7.48582930091083 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-or", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [ - { - "id": "11808", - "score": 7.48582930091083 - }, - { - "id": "11803", - "score": 7.48582930091083 - }, - { - "id": "11798", - "score": 7.48582930091083 - }, - { - "id": "11793", - "score": 7.48582930091083 - }, - { - "id": "11788", - "score": 7.48582930091083 - }, - { - "id": "11783", - "score": 7.48582930091083 - }, - { - "id": "11778", - "score": 7.48582930091083 - }, - { - "id": "11773", - "score": 7.48582930091083 - }, - { - "id": "11768", - "score": 7.48582930091083 - }, - { - "id": "11763", - "score": 7.48582930091083 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-paraphrase", - "relevant": { - "11511": 2, - "11512": 1 - }, - "hits": [], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "maple-absent", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "maple-near-miss", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "вектор-words", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11940", - "score": 7.931367874145508, - "lexicalRank": 35 - }, - { - "id": "11835", - "score": 7.92194128036499, - "lexicalRank": 56 - }, - { - "id": "11945", - "score": 7.5705389976501465, - "lexicalRank": 34 - }, - { - "id": "12015", - "score": 7.566339015960693, - "lexicalRank": 20 - }, - { - "id": "11910", - "score": 7.426838397979736, - "lexicalRank": 41 - }, - { - "id": "11990", - "score": 7.4217610359191895, - "lexicalRank": 25 - }, - { - "id": "11830", - "score": 7.3859992027282715, - "lexicalRank": 57 - }, - { - "id": "11915", - "score": 7.373218059539795, - "lexicalRank": 40 - }, - { - "id": "12075", - "score": 7.369237899780273, - "lexicalRank": 8 - }, - { - "id": "12035", - "score": 7.345060348510742, - "lexicalRank": 16 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - } - }, - { - "id": "вектор-typo", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11995", - "score": 6.678157329559326, - "lexicalRank": 24 - }, - { - "id": "11813", - "score": 6.626920223236084, - "lexicalRank": 101 - }, - { - "id": "12075", - "score": 6.614440441131592, - "lexicalRank": 7 - }, - { - "id": "11840", - "score": 6.4451212882995605, - "lexicalRank": 55 - }, - { - "id": "11970", - "score": 6.295319557189941, - "lexicalRank": 29 - }, - { - "id": "11890", - "score": 6.290195465087891, - "lexicalRank": 45 - }, - { - "id": "12040", - "score": 6.255374431610107, - "lexicalRank": 14 - }, - { - "id": "11820", - "score": 6.2226386070251465, - "lexicalRank": 59 - }, - { - "id": "11934", - "score": 6.207514762878418, - "lexicalRank": 218 - }, - { - "id": "12060", - "score": 6.2029337882995605, - "lexicalRank": 10 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 0.5, - "ndcgAt10": 0.52129602861432, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "вектор-beginnings", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11989", - "score": 3.2008016109466553, - "lexicalRank": 207 - }, - { - "id": "12019", - "score": 3.1179652214050293, - "lexicalRank": 201 - }, - { - "id": "11969", - "score": 3.0732762813568115, - "lexicalRank": 211 - }, - { - "id": "11889", - "score": 2.985116481781006, - "lexicalRank": 227 - }, - { - "id": "12004", - "score": 2.9849021434783936, - "lexicalRank": 204 - }, - { - "id": "12029", - "score": 2.914386749267578, - "lexicalRank": 199 - }, - { - "id": "11939", - "score": 2.819045066833496, - "lexicalRank": 217 - }, - { - "id": "11904", - "score": 2.8098416328430176, - "lexicalRank": 224 - }, - { - "id": "11924", - "score": 2.808605909347534, - "lexicalRank": 220 - }, - { - "id": "11934", - "score": 2.789806365966797, - "lexicalRank": 218 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - } - }, - { - "id": "вектор-phrase", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "score": 3.126652600631594 - }, - { - "id": "12110", - "score": 2.7623348535297314 - }, - { - "id": "12105", - "score": 2.7623348535297314 - }, - { - "id": "12100", - "score": 2.7623348535297314 - }, - { - "id": "12095", - "score": 2.7623348535297314 - }, - { - "id": "12090", - "score": 2.7623348535297314 - }, - { - "id": "12085", - "score": 2.7623348535297314 - }, - { - "id": "12080", - "score": 2.7623348535297314 - }, - { - "id": "12075", - "score": 2.7623348535297314 - }, - { - "id": "12070", - "score": 2.7623348535297314 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - } - }, - { - "id": "вектор-from", - "relevant": { - "11813": 2 - }, - "hits": [ - { - "id": "11813", - "score": -4.053653717041016, - "lexicalRank": 0 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - } - }, - { - "id": "вектор-chat", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "12043", - "score": -0.7538442611694336, - "lexicalRank": 96 - }, - { - "id": "12033", - "score": -0.8733174204826355, - "lexicalRank": 98 - }, - { - "id": "12028", - "score": -0.9477561116218567, - "lexicalRank": 99 - }, - { - "id": "12023", - "score": -1.0663375854492188, - "lexicalRank": 100 - }, - { - "id": "12083", - "score": -1.1414753198623657, - "lexicalRank": 88 - }, - { - "id": "12018", - "score": -1.2144559621810913, - "lexicalRank": 101 - }, - { - "id": "12038", - "score": -1.4278435707092285, - "lexicalRank": 97 - }, - { - "id": "12103", - "score": -1.4810831546783447, - "lexicalRank": 84 - }, - { - "id": "12068", - "score": -1.4991815090179443, - "lexicalRank": 91 - }, - { - "id": "12113", - "score": -1.5094108581542969, - "lexicalRank": 82 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - } - }, - { - "id": "вектор-date", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11813", - "score": -5.503037452697754, - "lexicalRank": 1 - }, - { - "id": "11814", - "score": -5.587318420410156, - "lexicalRank": 0 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - } - }, - { - "id": "вектор-and", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "score": 8.47765579013484 - }, - { - "id": "12110", - "score": 7.489838832938161 - }, - { - "id": "12105", - "score": 7.489838832938161 - }, - { - "id": "12100", - "score": 7.489838832938161 - }, - { - "id": "12095", - "score": 7.489838832938161 - }, - { - "id": "12090", - "score": 7.489838832938161 - }, - { - "id": "12085", - "score": 7.489838832938161 - }, - { - "id": "12080", - "score": 7.489838832938161 - }, - { - "id": "12075", - "score": 7.489838832938161 - }, - { - "id": "12070", - "score": 7.489838832938161 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - } - }, - { - "id": "вектор-or", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [ - { - "id": "11814", - "score": 8.47765579013484 - }, - { - "id": "12110", - "score": 7.489838832938161 - }, - { - "id": "12105", - "score": 7.489838832938161 - }, - { - "id": "12100", - "score": 7.489838832938161 - }, - { - "id": "12095", - "score": 7.489838832938161 - }, - { - "id": "12090", - "score": 7.489838832938161 - }, - { - "id": "12085", - "score": 7.489838832938161 - }, - { - "id": "12080", - "score": 7.489838832938161 - }, - { - "id": "12075", - "score": 7.489838832938161 - }, - { - "id": "12070", - "score": 7.489838832938161 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - } - }, - { - "id": "вектор-paraphrase", - "relevant": { - "11813": 2, - "11814": 1 - }, - "hits": [], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "вектор-absent", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "вектор-near-miss", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "prism-words", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12115", - "score": 8.16292667388916, - "lexicalRank": 61 - }, - { - "id": "12116", - "score": 8.108729362487793, - "lexicalRank": 60 - }, - { - "id": "12350", - "score": 6.787620544433594, - "lexicalRank": 255 - }, - { - "id": "12280", - "score": 6.751938819885254, - "lexicalRank": 269 - }, - { - "id": "12310", - "score": 6.689011573791504, - "lexicalRank": 263 - }, - { - "id": "12300", - "score": 6.637208461761475, - "lexicalRank": 265 - }, - { - "id": "12355", - "score": 6.6052045822143555, - "lexicalRank": 254 - }, - { - "id": "12410", - "score": 6.5069379806518555, - "lexicalRank": 243 - }, - { - "id": "12240", - "score": 6.487910747528076, - "lexicalRank": 277 - }, - { - "id": "12200", - "score": 6.4574360847473145, - "lexicalRank": 285 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-typo", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12115", - "score": 7.70074987411499, - "lexicalRank": 61 - }, - { - "id": "12116", - "score": 7.129725933074951, - "lexicalRank": 60 - }, - { - "id": "12367", - "score": 5.932244777679443, - "lexicalRank": 9 - }, - { - "id": "12247", - "score": 5.629062175750732, - "lexicalRank": 33 - }, - { - "id": "12365", - "score": 5.586015701293945, - "lexicalRank": 252 - }, - { - "id": "12317", - "score": 5.5220842361450195, - "lexicalRank": 19 - }, - { - "id": "12177", - "score": 5.4242119789123535, - "lexicalRank": 47 - }, - { - "id": "12172", - "score": 5.330693244934082, - "lexicalRank": 48 - }, - { - "id": "12385", - "score": 5.287479877471924, - "lexicalRank": 248 - }, - { - "id": "12202", - "score": 5.262482166290283, - "lexicalRank": 42 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-beginnings", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12115", - "score": 8.649856567382812, - "lexicalRank": 61 - }, - { - "id": "12116", - "score": 7.992773532867432, - "lexicalRank": 60 - }, - { - "id": "12357", - "score": 6.6164374351501465, - "lexicalRank": 11 - }, - { - "id": "12402", - "score": 6.311557769775391, - "lexicalRank": 2 - }, - { - "id": "12367", - "score": 6.110666751861572, - "lexicalRank": 9 - }, - { - "id": "12272", - "score": 6.051764965057373, - "lexicalRank": 28 - }, - { - "id": "12317", - "score": 6.044123649597168, - "lexicalRank": 19 - }, - { - "id": "12117", - "score": 6.029598236083984, - "lexicalRank": 59 - }, - { - "id": "12142", - "score": 6.0267720222473145, - "lexicalRank": 54 - }, - { - "id": "12132", - "score": 5.9389519691467285, - "lexicalRank": 56 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-phrase", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "score": 2.7623348535297314 - }, - { - "id": "12407", - "score": 2.7623348535297314 - }, - { - "id": "12402", - "score": 2.7623348535297314 - }, - { - "id": "12397", - "score": 2.7623348535297314 - }, - { - "id": "12392", - "score": 2.7623348535297314 - }, - { - "id": "12387", - "score": 2.7623348535297314 - }, - { - "id": "12382", - "score": 2.7623348535297314 - }, - { - "id": "12377", - "score": 2.7623348535297314 - }, - { - "id": "12372", - "score": 2.7623348535297314 - }, - { - "id": "12367", - "score": 2.7623348535297314 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-from", - "relevant": { - "12115": 2 - }, - "hits": [ - { - "id": "12115", - "score": -3.4879183769226074, - "lexicalRank": 0 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - } - }, - { - "id": "prism-chat", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12345", - "score": -0.1605147272348404, - "lexicalRank": 96 - }, - { - "id": "12325", - "score": -0.25931647419929504, - "lexicalRank": 100 - }, - { - "id": "12340", - "score": -0.46640682220458984, - "lexicalRank": 97 - }, - { - "id": "12365", - "score": -0.5419776439666748, - "lexicalRank": 92 - }, - { - "id": "12355", - "score": -0.5792048573493958, - "lexicalRank": 94 - }, - { - "id": "12405", - "score": -0.6089483499526978, - "lexicalRank": 84 - }, - { - "id": "12380", - "score": -0.6718494296073914, - "lexicalRank": 89 - }, - { - "id": "12395", - "score": -0.6834031343460083, - "lexicalRank": 86 - }, - { - "id": "12415", - "score": -0.6878743767738342, - "lexicalRank": 82 - }, - { - "id": "12370", - "score": -0.705737829208374, - "lexicalRank": 91 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-date", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12116", - "score": -3.8352277278900146, - "lexicalRank": 0 - }, - { - "id": "12115", - "score": -5.049022197723389, - "lexicalRank": 1 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - } - }, - { - "id": "prism-and", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "score": 7.489838832938161 - }, - { - "id": "12407", - "score": 7.489838832938161 - }, - { - "id": "12402", - "score": 7.489838832938161 - }, - { - "id": "12397", - "score": 7.489838832938161 - }, - { - "id": "12392", - "score": 7.489838832938161 - }, - { - "id": "12387", - "score": 7.489838832938161 - }, - { - "id": "12382", - "score": 7.489838832938161 - }, - { - "id": "12377", - "score": 7.489838832938161 - }, - { - "id": "12372", - "score": 7.489838832938161 - }, - { - "id": "12367", - "score": 7.489838832938161 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-or", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [ - { - "id": "12412", - "score": 7.489838832938161 - }, - { - "id": "12407", - "score": 7.489838832938161 - }, - { - "id": "12402", - "score": 7.489838832938161 - }, - { - "id": "12397", - "score": 7.489838832938161 - }, - { - "id": "12392", - "score": 7.489838832938161 - }, - { - "id": "12387", - "score": 7.489838832938161 - }, - { - "id": "12382", - "score": 7.489838832938161 - }, - { - "id": "12377", - "score": 7.489838832938161 - }, - { - "id": "12372", - "score": 7.489838832938161 - }, - { - "id": "12367", - "score": 7.489838832938161 - } - ], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-paraphrase", - "relevant": { - "12115": 2, - "12116": 1 - }, - "hits": [], - "metrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - } - }, - { - "id": "prism-absent", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "prism-near-miss", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - } - ] - }, - "questions": { - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.6875, - "mrr": 1, - "ndcgAt10": 0.8659851083291633, - "noAnswerFalseHits": 0 - }, - "rows": [ - { - "id": "cobalt-question-decision", - "relevant": { - "70001": 2, - "70002": 1 - }, - "hits": [ - { - "id": "70001", - "score": 10.751850128173828, - "lexicalRank": 121 - }, - { - "id": "70002", - "score": 8.593709945678711, - "lexicalRank": 60 - }, - { - "id": "70140", - "score": 5.350794315338135, - "lexicalRank": 32 - }, - { - "id": "70055", - "score": 5.1161675453186035, - "lexicalRank": 49 - }, - { - "id": "70115", - "score": 5.016029357910156, - "lexicalRank": 37 - }, - { - "id": "70030", - "score": 4.977260112762451, - "lexicalRank": 54 - }, - { - "id": "70045", - "score": 4.861608028411865, - "lexicalRank": 51 - }, - { - "id": "70100", - "score": 4.83193302154541, - "lexicalRank": 40 - }, - { - "id": "70090", - "score": 4.808356761932373, - "lexicalRank": 42 - }, - { - "id": "70125", - "score": 4.754176616668701, - "lexicalRank": 35 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - } - }, - { - "id": "cobalt-question-question", - "relevant": { - "70001": 2, - "70002": 1 - }, - "hits": [ - { - "id": "70001", - "score": 7.23106575012207, - "lexicalRank": 121 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - } - }, - { - "id": "cobalt-question-no-answer", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "birch-question-decision", - "relevant": { - "70153": 2, - "70154": 1 - }, - "hits": [ - { - "id": "70153", - "score": 10.18606185913086, - "lexicalRank": 30 - }, - { - "id": "70154", - "score": 9.900846481323242, - "lexicalRank": 61 - }, - { - "id": "70197", - "score": 5.7486066818237305, - "lexicalRank": 52 - }, - { - "id": "70242", - "score": 5.041281700134277, - "lexicalRank": 43 - }, - { - "id": "70287", - "score": 4.884515285491943, - "lexicalRank": 34 - }, - { - "id": "70277", - "score": 4.856664657592773, - "lexicalRank": 36 - }, - { - "id": "70212", - "score": 4.822803020477295, - "lexicalRank": 49 - }, - { - "id": "70207", - "score": 4.788071632385254, - "lexicalRank": 50 - }, - { - "id": "70182", - "score": 4.78555154800415, - "lexicalRank": 55 - }, - { - "id": "70237", - "score": 4.770137786865234, - "lexicalRank": 44 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - } - }, - { - "id": "birch-question-question", - "relevant": { - "70153": 2, - "70154": 1 - }, - "hits": [ - { - "id": "70153", - "score": 10.70551586151123, - "lexicalRank": 30 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - } - }, - { - "id": "birch-question-no-answer", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "спектр-question-decision", - "relevant": { - "70305": 2, - "70306": 1 - }, - "hits": [ - { - "id": "70305", - "score": 9.470867156982422, - "lexicalRank": 31 - }, - { - "id": "70449", - "score": 4.701529502868652, - "lexicalRank": 33 - }, - { - "id": "70429", - "score": 4.574463844299316, - "lexicalRank": 37 - }, - { - "id": "70379", - "score": 4.497837543487549, - "lexicalRank": 47 - }, - { - "id": "70409", - "score": 4.488161563873291, - "lexicalRank": 41 - }, - { - "id": "70342", - "score": 4.469592094421387, - "lexicalRank": 114 - }, - { - "id": "70439", - "score": 4.46698522567749, - "lexicalRank": 35 - }, - { - "id": "70367", - "score": 4.462706089019775, - "lexicalRank": 109 - }, - { - "id": "70454", - "score": 4.4601030349731445, - "lexicalRank": 32 - }, - { - "id": "70414", - "score": 4.357937812805176, - "lexicalRank": 40 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - } - }, - { - "id": "спектр-question-question", - "relevant": { - "70305": 2, - "70306": 1 - }, - "hits": [ - { - "id": "70305", - "score": 7.549112319946289, - "lexicalRank": 31 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - } - }, - { - "id": "спектр-question-no-answer", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - }, - { - "id": "nova-question-decision", - "relevant": { - "70457": 2, - "70458": 1 - }, - "hits": [ - { - "id": "70458", - "score": 8.890884399414062, - "lexicalRank": 60 - }, - { - "id": "70457", - "score": 8.344308853149414, - "lexicalRank": 61 - }, - { - "id": "70581", - "score": 4.407783508300781, - "lexicalRank": 35 - }, - { - "id": "70526", - "score": 4.299479007720947, - "lexicalRank": 46 - }, - { - "id": "70556", - "score": 4.249410152435303, - "lexicalRank": 40 - }, - { - "id": "70566", - "score": 4.116221904754639, - "lexicalRank": 38 - }, - { - "id": "70501", - "score": 4.0909576416015625, - "lexicalRank": 51 - }, - { - "id": "70531", - "score": 3.9745075702667236, - "lexicalRank": 45 - }, - { - "id": "70516", - "score": 3.9253413677215576, - "lexicalRank": 48 - }, - { - "id": "70586", - "score": 3.8646199703216553, - "lexicalRank": 34 - } - ], - "metrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - } - }, - { - "id": "nova-question-question", - "relevant": { - "70457": 2, - "70458": 1 - }, - "hits": [ - { - "id": "70457", - "score": 10.7865629196167, - "lexicalRank": 61 - } - ], - "metrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - } - }, - { - "id": "nova-question-no-answer", - "relevant": {}, - "hits": [], - "metrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - } - } - ] - } - }, - "inference": { - "pairInferenceMs": 136790.54689099995, - "scoredPairs": 13464, - "averagePairMs": 10.159725704916811, - "loadMs": 1173.3451759999998, - "embeddedMessages": 0, - "uniqueEmbeddedQueries": 0, - "uniqueCrossEncoderQueries": 72, - "passageInferenceMs": 0, - "queryInferenceMs": 0, - "amortizedPerMessageMs": null, - "cache": "exact synthetic query-message score cache, empty at run start" - }, - "candidateRows": [ - { - "id": "aurora-words", - "category": "words", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-words", - "category": "words", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-words", - "category": "words", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-words", - "category": "words", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 50, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 100, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 300, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 500, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-and", - "category": "and", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-and", - "category": "and", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-and", - "category": "and", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-and", - "category": "and", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-or", - "category": "or", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-or", - "category": "or", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-or", - "category": "or", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-or", - "category": "or", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-words", - "category": "words", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-words", - "category": "words", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-words", - "category": "words", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-words", - "category": "words", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 50, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 100, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 300, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 500, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-and", - "category": "and", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-and", - "category": "and", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-and", - "category": "and", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-and", - "category": "and", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-or", - "category": "or", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-or", - "category": "or", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-or", - "category": "or", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-or", - "category": "or", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-words", - "category": "words", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-words", - "category": "words", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-words", - "category": "words", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-words", - "category": "words", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662" - ], - "candidateRecall": 0.5, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10605", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10605", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611" - ], - "candidateRecall": 1, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10605", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611" - ], - "candidateRecall": 1, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 50, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 100, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 300, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 500, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-and", - "category": "and", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-and", - "category": "and", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-and", - "category": "and", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-and", - "category": "and", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-or", - "category": "or", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-or", - "category": "or", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-or", - "category": "or", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-or", - "category": "or", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-words", - "category": "words", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-words", - "category": "words", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-words", - "category": "words", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-words", - "category": "words", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 50, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 100, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 300, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 500, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-and", - "category": "and", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-and", - "category": "and", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-and", - "category": "and", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-and", - "category": "and", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-or", - "category": "or", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-or", - "category": "or", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-or", - "category": "or", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-or", - "category": "or", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - } - ], - "keywordRows": [ - { - "id": "aurora-words", - "category": "words", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10001", - "score": 9.144113540649414, - "lexicalRank": 60 - }, - { - "id": "10002", - "score": 8.763883590698242, - "lexicalRank": 121 - }, - { - "id": "10046", - "score": 8.404559135437012, - "lexicalRank": 293 - }, - { - "id": "10211", - "score": 8.33627986907959, - "lexicalRank": 260 - }, - { - "id": "10086", - "score": 8.127921104431152, - "lexicalRank": 285 - }, - { - "id": "10186", - "score": 8.10696029663086, - "lexicalRank": 265 - }, - { - "id": "10236", - "score": 8.104728698730469, - "lexicalRank": 255 - }, - { - "id": "10191", - "score": 8.0871000289917, - "lexicalRank": 264 - }, - { - "id": "10146", - "score": 8.065347671508789, - "lexicalRank": 273 - }, - { - "id": "10116", - "score": 8.039112091064453, - "lexicalRank": 279 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-typo", - "category": "typo", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10002", - "score": 8.407031059265137, - "lexicalRank": 121 - }, - { - "id": "10001", - "score": 7.457723617553711, - "lexicalRank": 60 - }, - { - "id": "10090", - "score": 7.038037300109863, - "lexicalRank": 164 - }, - { - "id": "10218", - "score": 7.009069919586182, - "lexicalRank": 16 - }, - { - "id": "10055", - "score": 6.976167678833008, - "lexicalRank": 171 - }, - { - "id": "10233", - "score": 6.961915969848633, - "lexicalRank": 13 - }, - { - "id": "10153", - "score": 6.896630764007568, - "lexicalRank": 29 - }, - { - "id": "10247", - "score": 6.875993251800537, - "lexicalRank": 193 - }, - { - "id": "10223", - "score": 6.869973659515381, - "lexicalRank": 15 - }, - { - "id": "10015", - "score": 6.8225297927856445, - "lexicalRank": 179 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10002", - "score": 8.832911491394043, - "lexicalRank": 121 - }, - { - "id": "10001", - "score": 8.565581321716309, - "lexicalRank": 60 - }, - { - "id": "10055", - "score": 7.864835739135742, - "lexicalRank": 171 - }, - { - "id": "10205", - "score": 7.708717346191406, - "lexicalRank": 141 - }, - { - "id": "10175", - "score": 7.707800388336182, - "lexicalRank": 147 - }, - { - "id": "10155", - "score": 7.70137357711792, - "lexicalRank": 151 - }, - { - "id": "10150", - "score": 7.671929836273193, - "lexicalRank": 152 - }, - { - "id": "10065", - "score": 7.642721176147461, - "lexicalRank": 169 - }, - { - "id": "10210", - "score": 7.6175618171691895, - "lexicalRank": 140 - }, - { - "id": "10070", - "score": 7.612574100494385, - "lexicalRank": 168 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-phrase", - "category": "phrase", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 2.7623348535297314 - }, - { - "id": "10293", - "score": 2.7623348535297314 - }, - { - "id": "10288", - "score": 2.7623348535297314 - }, - { - "id": "10283", - "score": 2.7623348535297314 - }, - { - "id": "10278", - "score": 2.7623348535297314 - }, - { - "id": "10273", - "score": 2.7623348535297314 - }, - { - "id": "10268", - "score": 2.7623348535297314 - }, - { - "id": "10263", - "score": 2.7623348535297314 - }, - { - "id": "10258", - "score": 2.7623348535297314 - }, - { - "id": "10253", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "10298", - "score": 2.7623348535297314 - }, - { - "id": "10293", - "score": 2.7623348535297314 - }, - { - "id": "10288", - "score": 2.7623348535297314 - }, - { - "id": "10283", - "score": 2.7623348535297314 - }, - { - "id": "10278", - "score": 2.7623348535297314 - }, - { - "id": "10273", - "score": 2.7623348535297314 - }, - { - "id": "10268", - "score": 2.7623348535297314 - }, - { - "id": "10263", - "score": 2.7623348535297314 - }, - { - "id": "10258", - "score": 2.7623348535297314 - }, - { - "id": "10253", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "aurora-from", - "category": "from", - "relevant": { - "10001": 2 - }, - "lexical": [ - { - "id": "10001", - "score": 1 - } - ], - "semantic": [ - { - "id": "10001", - "score": -2.133422374725342, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-chat", - "category": "chat", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10206", - "score": 1.1320911645889282, - "lexicalRank": 101 - }, - { - "id": "10226", - "score": 1.127381682395935, - "lexicalRank": 97 - }, - { - "id": "10291", - "score": 1.0891575813293457, - "lexicalRank": 84 - }, - { - "id": "10221", - "score": 0.7220429182052612, - "lexicalRank": 98 - }, - { - "id": "10246", - "score": 0.6760910153388977, - "lexicalRank": 93 - }, - { - "id": "10296", - "score": 0.6750655770301819, - "lexicalRank": 83 - }, - { - "id": "10281", - "score": 0.6719631552696228, - "lexicalRank": 86 - }, - { - "id": "10231", - "score": 0.6679152250289917, - "lexicalRank": 96 - }, - { - "id": "10271", - "score": 0.654645562171936, - "lexicalRank": 88 - }, - { - "id": "10216", - "score": 0.5903054475784302, - "lexicalRank": 99 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-date", - "category": "date", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10001", - "score": 1 - }, - { - "id": "10002", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10001", - "score": -2.672081708908081, - "lexicalRank": 0 - }, - { - "id": "10002", - "score": -3.1790313720703125, - "lexicalRank": 1 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-and", - "category": "and", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "aurora-or", - "category": "or", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-words", - "category": "words", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10304", - "score": 8.97688102722168, - "lexicalRank": 121 - }, - { - "id": "10598", - "score": 7.926235198974609, - "lexicalRank": 243 - }, - { - "id": "10303", - "score": 7.766183376312256, - "lexicalRank": 60 - }, - { - "id": "10453", - "score": 7.723439693450928, - "lexicalRank": 272 - }, - { - "id": "10543", - "score": 7.701955795288086, - "lexicalRank": 254 - }, - { - "id": "10603", - "score": 7.637665748596191, - "lexicalRank": 242 - }, - { - "id": "10413", - "score": 7.637113094329834, - "lexicalRank": 280 - }, - { - "id": "10488", - "score": 7.61979866027832, - "lexicalRank": 265 - }, - { - "id": "10478", - "score": 7.59701681137085, - "lexicalRank": 267 - }, - { - "id": "10458", - "score": 7.580545902252197, - "lexicalRank": 271 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.6885288809404666, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-typo", - "category": "typo", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10304", - "score": 4.184720516204834, - "lexicalRank": 121 - }, - { - "id": "10303", - "score": 3.1427712440490723, - "lexicalRank": 60 - }, - { - "id": "10590", - "score": 2.2103934288024902, - "lexicalRank": 2 - }, - { - "id": "10547", - "score": 2.043490171432495, - "lexicalRank": 133 - }, - { - "id": "10347", - "score": 1.9820915460586548, - "lexicalRank": 173 - }, - { - "id": "10572", - "score": 1.9670615196228027, - "lexicalRank": 128 - }, - { - "id": "10372", - "score": 1.941902756690979, - "lexicalRank": 168 - }, - { - "id": "10467", - "score": 1.9225600957870483, - "lexicalRank": 149 - }, - { - "id": "10452", - "score": 1.9188569784164429, - "lexicalRank": 152 - }, - { - "id": "10482", - "score": 1.9111326932907104, - "lexicalRank": 146 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10304", - "score": 8.845378875732422, - "lexicalRank": 121 - }, - { - "id": "10303", - "score": 7.421807765960693, - "lexicalRank": 60 - }, - { - "id": "10548", - "score": 6.571941375732422, - "lexicalRank": 253 - }, - { - "id": "10503", - "score": 6.559657573699951, - "lexicalRank": 262 - }, - { - "id": "10433", - "score": 6.476851940155029, - "lexicalRank": 276 - }, - { - "id": "10378", - "score": 6.4482622146606445, - "lexicalRank": 287 - }, - { - "id": "10518", - "score": 6.384768962860107, - "lexicalRank": 259 - }, - { - "id": "10368", - "score": 6.379504680633545, - "lexicalRank": 289 - }, - { - "id": "10568", - "score": 6.364409923553467, - "lexicalRank": 249 - }, - { - "id": "10598", - "score": 6.302361488342285, - "lexicalRank": 243 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-phrase", - "category": "phrase", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 2.7623348535297314 - }, - { - "id": "10595", - "score": 2.7623348535297314 - }, - { - "id": "10590", - "score": 2.7623348535297314 - }, - { - "id": "10585", - "score": 2.7623348535297314 - }, - { - "id": "10580", - "score": 2.7623348535297314 - }, - { - "id": "10575", - "score": 2.7623348535297314 - }, - { - "id": "10570", - "score": 2.7623348535297314 - }, - { - "id": "10565", - "score": 2.7623348535297314 - }, - { - "id": "10560", - "score": 2.7623348535297314 - }, - { - "id": "10555", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "10600", - "score": 2.7623348535297314 - }, - { - "id": "10595", - "score": 2.7623348535297314 - }, - { - "id": "10590", - "score": 2.7623348535297314 - }, - { - "id": "10585", - "score": 2.7623348535297314 - }, - { - "id": "10580", - "score": 2.7623348535297314 - }, - { - "id": "10575", - "score": 2.7623348535297314 - }, - { - "id": "10570", - "score": 2.7623348535297314 - }, - { - "id": "10565", - "score": 2.7623348535297314 - }, - { - "id": "10560", - "score": 2.7623348535297314 - }, - { - "id": "10555", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "cedar-from", - "category": "from", - "relevant": { - "10303": 2 - }, - "lexical": [ - { - "id": "10303", - "score": 1 - } - ], - "semantic": [ - { - "id": "10303", - "score": -2.1878979206085205, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-chat", - "category": "chat", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10548", - "score": -0.05690937489271164, - "lexicalRank": 93 - }, - { - "id": "10593", - "score": -0.1841631978750229, - "lexicalRank": 84 - }, - { - "id": "10518", - "score": -0.2792830169200897, - "lexicalRank": 99 - }, - { - "id": "10598", - "score": -0.36233586072921753, - "lexicalRank": 83 - }, - { - "id": "10558", - "score": -0.36453109979629517, - "lexicalRank": 91 - }, - { - "id": "10513", - "score": -0.41426771879196167, - "lexicalRank": 100 - }, - { - "id": "10508", - "score": -0.42688512802124023, - "lexicalRank": 101 - }, - { - "id": "10304", - "score": -0.5078280568122864, - "lexicalRank": 41 - }, - { - "id": "10538", - "score": -0.5147302150726318, - "lexicalRank": 95 - }, - { - "id": "10573", - "score": -0.5198200345039368, - "lexicalRank": 88 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0.5, - "mrr": 0.125, - "ndcgAt10": 0.08688267143572001, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-date", - "category": "date", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10303", - "score": 1 - }, - { - "id": "10304", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10304", - "score": -1.9164950847625732, - "lexicalRank": 1 - }, - { - "id": "10303", - "score": -2.703486919403076, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-and", - "category": "and", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "cedar-or", - "category": "or", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-words", - "category": "words", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10605", - "score": 9.941312789916992, - "lexicalRank": 1 - }, - { - "id": "10807", - "score": 8.448219299316406, - "lexicalRank": 21 - }, - { - "id": "10822", - "score": 8.419659614562988, - "lexicalRank": 18 - }, - { - "id": "10632", - "score": 8.358668327331543, - "lexicalRank": 56 - }, - { - "id": "10767", - "score": 8.349440574645996, - "lexicalRank": 29 - }, - { - "id": "10742", - "score": 8.334734916687012, - "lexicalRank": 34 - }, - { - "id": "10847", - "score": 8.306585311889648, - "lexicalRank": 13 - }, - { - "id": "10852", - "score": 8.301280975341797, - "lexicalRank": 12 - }, - { - "id": "10837", - "score": 8.297823905944824, - "lexicalRank": 15 - }, - { - "id": "10797", - "score": 8.287442207336426, - "lexicalRank": 23 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-typo", - "category": "typo", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10801", - "score": 5.321130752563477, - "lexicalRank": 203 - }, - { - "id": "10866", - "score": 5.154948711395264, - "lexicalRank": 190 - }, - { - "id": "10721", - "score": 5.130404949188232, - "lexicalRank": 219 - }, - { - "id": "10726", - "score": 5.04014253616333, - "lexicalRank": 218 - }, - { - "id": "10711", - "score": 5.0147480964660645, - "lexicalRank": 221 - }, - { - "id": "10671", - "score": 4.996459484100342, - "lexicalRank": 229 - }, - { - "id": "10856", - "score": 4.986394882202148, - "lexicalRank": 192 - }, - { - "id": "10781", - "score": 4.971275329589844, - "lexicalRank": 207 - }, - { - "id": "10681", - "score": 4.969725608825684, - "lexicalRank": 227 - }, - { - "id": "10811", - "score": 4.945552349090576, - "lexicalRank": 201 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10605", - "score": 8.343690872192383, - "lexicalRank": 1 - }, - { - "id": "10632", - "score": 6.755748748779297, - "lexicalRank": 56 - }, - { - "id": "10727", - "score": 6.705746173858643, - "lexicalRank": 37 - }, - { - "id": "10607", - "score": 6.353328227996826, - "lexicalRank": 61 - }, - { - "id": "10747", - "score": 6.205280303955078, - "lexicalRank": 33 - }, - { - "id": "10692", - "score": 6.176116466522217, - "lexicalRank": 44 - }, - { - "id": "10667", - "score": 6.106525421142578, - "lexicalRank": 49 - }, - { - "id": "10657", - "score": 6.102047443389893, - "lexicalRank": 51 - }, - { - "id": "10736", - "score": 6.071948051452637, - "lexicalRank": 216 - }, - { - "id": "10622", - "score": 6.069979190826416, - "lexicalRank": 58 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-phrase", - "category": "phrase", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 3.126652600631594 - }, - { - "id": "10902", - "score": 2.7623348535297314 - }, - { - "id": "10897", - "score": 2.7623348535297314 - }, - { - "id": "10892", - "score": 2.7623348535297314 - }, - { - "id": "10887", - "score": 2.7623348535297314 - }, - { - "id": "10882", - "score": 2.7623348535297314 - }, - { - "id": "10877", - "score": 2.7623348535297314 - }, - { - "id": "10872", - "score": 2.7623348535297314 - }, - { - "id": "10867", - "score": 2.7623348535297314 - }, - { - "id": "10862", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "10606", - "score": 3.126652600631594 - }, - { - "id": "10902", - "score": 2.7623348535297314 - }, - { - "id": "10897", - "score": 2.7623348535297314 - }, - { - "id": "10892", - "score": 2.7623348535297314 - }, - { - "id": "10887", - "score": 2.7623348535297314 - }, - { - "id": "10882", - "score": 2.7623348535297314 - }, - { - "id": "10877", - "score": 2.7623348535297314 - }, - { - "id": "10872", - "score": 2.7623348535297314 - }, - { - "id": "10867", - "score": 2.7623348535297314 - }, - { - "id": "10862", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "север-from", - "category": "from", - "relevant": { - "10605": 2 - }, - "lexical": [ - { - "id": "10605", - "score": 1 - } - ], - "semantic": [ - { - "id": "10605", - "score": -1.4385977983474731, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-chat", - "category": "chat", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10605", - "score": 1 - }, - { - "id": "10606", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10810", - "score": -0.8035014867782593, - "lexicalRank": 101 - }, - { - "id": "10870", - "score": -0.8724918365478516, - "lexicalRank": 89 - }, - { - "id": "10820", - "score": -0.8758571147918701, - "lexicalRank": 99 - }, - { - "id": "10890", - "score": -0.9417440295219421, - "lexicalRank": 85 - }, - { - "id": "10900", - "score": -1.0117398500442505, - "lexicalRank": 83 - }, - { - "id": "10815", - "score": -1.0464911460876465, - "lexicalRank": 100 - }, - { - "id": "10825", - "score": -1.126318097114563, - "lexicalRank": 98 - }, - { - "id": "10855", - "score": -1.2063350677490234, - "lexicalRank": 92 - }, - { - "id": "10850", - "score": -1.2222322225570679, - "lexicalRank": 93 - }, - { - "id": "10880", - "score": -1.280145287513733, - "lexicalRank": 87 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-date", - "category": "date", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10605", - "score": 1 - }, - { - "id": "10606", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10605", - "score": 0.5484523177146912, - "lexicalRank": 0 - }, - { - "id": "10606", - "score": -3.554842233657837, - "lexicalRank": 1 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-and", - "category": "and", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "север-or", - "category": "or", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-words", - "category": "words", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10907", - "score": 9.74099349975586, - "lexicalRank": 61 - }, - { - "id": "10908", - "score": 9.090091705322266, - "lexicalRank": 60 - }, - { - "id": "11062", - "score": 6.556122779846191, - "lexicalRank": 271 - }, - { - "id": "11057", - "score": 6.3410139083862305, - "lexicalRank": 272 - }, - { - "id": "10942", - "score": 6.337686061859131, - "lexicalRank": 295 - }, - { - "id": "11052", - "score": 6.325874328613281, - "lexicalRank": 273 - }, - { - "id": "10922", - "score": 6.1378703117370605, - "lexicalRank": 299 - }, - { - "id": "11087", - "score": 6.117567539215088, - "lexicalRank": 266 - }, - { - "id": "11012", - "score": 6.073670864105225, - "lexicalRank": 281 - }, - { - "id": "10937", - "score": 6.069832801818848, - "lexicalRank": 296 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-typo", - "category": "typo", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10907", - "score": 9.766773223876953, - "lexicalRank": 61 - }, - { - "id": "10908", - "score": 8.03225326538086, - "lexicalRank": 60 - }, - { - "id": "11029", - "score": 6.531184673309326, - "lexicalRank": 35 - }, - { - "id": "11154", - "score": 6.201030254364014, - "lexicalRank": 10 - }, - { - "id": "10949", - "score": 6.130903244018555, - "lexicalRank": 51 - }, - { - "id": "11136", - "score": 6.114521026611328, - "lexicalRank": 136 - }, - { - "id": "10979", - "score": 5.984946250915527, - "lexicalRank": 45 - }, - { - "id": "10987", - "score": 5.975259304046631, - "lexicalRank": 286 - }, - { - "id": "11081", - "score": 5.9714884757995605, - "lexicalRank": 147 - }, - { - "id": "11199", - "score": 5.929468631744385, - "lexicalRank": 1 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10907", - "score": 9.180591583251953, - "lexicalRank": 61 - }, - { - "id": "10908", - "score": 8.467389106750488, - "lexicalRank": 60 - }, - { - "id": "10909", - "score": 6.508573532104492, - "lexicalRank": 59 - }, - { - "id": "11086", - "score": 6.209899425506592, - "lexicalRank": 146 - }, - { - "id": "11101", - "score": 6.07896614074707, - "lexicalRank": 143 - }, - { - "id": "10954", - "score": 6.045144081115723, - "lexicalRank": 50 - }, - { - "id": "11084", - "score": 6.017730236053467, - "lexicalRank": 24 - }, - { - "id": "10947", - "score": 6.009658336639404, - "lexicalRank": 294 - }, - { - "id": "11102", - "score": 5.957398891448975, - "lexicalRank": 263 - }, - { - "id": "11132", - "score": 5.930818557739258, - "lexicalRank": 257 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-phrase", - "category": "phrase", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 2.7623348535297314 - }, - { - "id": "11199", - "score": 2.7623348535297314 - }, - { - "id": "11194", - "score": 2.7623348535297314 - }, - { - "id": "11189", - "score": 2.7623348535297314 - }, - { - "id": "11184", - "score": 2.7623348535297314 - }, - { - "id": "11179", - "score": 2.7623348535297314 - }, - { - "id": "11174", - "score": 2.7623348535297314 - }, - { - "id": "11169", - "score": 2.7623348535297314 - }, - { - "id": "11164", - "score": 2.7623348535297314 - }, - { - "id": "11159", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "11204", - "score": 2.7623348535297314 - }, - { - "id": "11199", - "score": 2.7623348535297314 - }, - { - "id": "11194", - "score": 2.7623348535297314 - }, - { - "id": "11189", - "score": 2.7623348535297314 - }, - { - "id": "11184", - "score": 2.7623348535297314 - }, - { - "id": "11179", - "score": 2.7623348535297314 - }, - { - "id": "11174", - "score": 2.7623348535297314 - }, - { - "id": "11169", - "score": 2.7623348535297314 - }, - { - "id": "11164", - "score": 2.7623348535297314 - }, - { - "id": "11159", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "lumen-from", - "category": "from", - "relevant": { - "10907": 2 - }, - "lexical": [ - { - "id": "10907", - "score": 1 - } - ], - "semantic": [ - { - "id": "10907", - "score": -1.6173491477966309, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-chat", - "category": "chat", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "11177", - "score": -1.4219605922698975, - "lexicalRank": 88 - }, - { - "id": "11122", - "score": -1.5253188610076904, - "lexicalRank": 99 - }, - { - "id": "11142", - "score": -1.5526238679885864, - "lexicalRank": 95 - }, - { - "id": "11172", - "score": -1.5592488050460815, - "lexicalRank": 89 - }, - { - "id": "11137", - "score": -1.5898562669754028, - "lexicalRank": 96 - }, - { - "id": "11132", - "score": -1.6688312292099, - "lexicalRank": 97 - }, - { - "id": "11192", - "score": -1.6710636615753174, - "lexicalRank": 85 - }, - { - "id": "11152", - "score": -1.6879723072052002, - "lexicalRank": 93 - }, - { - "id": "11162", - "score": -1.6941324472427368, - "lexicalRank": 91 - }, - { - "id": "11147", - "score": -1.7030967473983765, - "lexicalRank": 94 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-date", - "category": "date", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "10908", - "score": 1 - }, - { - "id": "10907", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10908", - "score": -3.8795433044433594, - "lexicalRank": 0 - }, - { - "id": "10907", - "score": -4.088683605194092, - "lexicalRank": 1 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-and", - "category": "and", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "lumen-or", - "category": "or", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - } - ], - "questionRows": [ - { - "id": "aurora-question-decision", - "category": "decision", - "relevant": { - "10001": 2, - "10002": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10001", - "score": 8.382225036621094, - "lexicalRank": 60 - }, - { - "id": "10302", - "score": 7.452626705169678, - "lexicalRank": 182 - }, - { - "id": "10022", - "score": 7.383761882781982, - "lexicalRank": 238 - }, - { - "id": "10122", - "score": 7.347390174865723, - "lexicalRank": 218 - }, - { - "id": "10297", - "score": 7.336249351501465, - "lexicalRank": 183 - }, - { - "id": "10172", - "score": 7.281764030456543, - "lexicalRank": 208 - }, - { - "id": "10167", - "score": 7.272385597229004, - "lexicalRank": 209 - }, - { - "id": "10067", - "score": 7.264976978302002, - "lexicalRank": 229 - }, - { - "id": "10097", - "score": 7.242596626281738, - "lexicalRank": 223 - }, - { - "id": "10192", - "score": 7.228582382202148, - "lexicalRank": 204 - } - ], - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "aurora-question-question", - "category": "question", - "relevant": { - "10001": 2, - "10002": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10001", - "score": 5.120489597320557, - "lexicalRank": 60 - }, - { - "id": "10002", - "score": 0.6442391872406006, - "lexicalRank": 121 - }, - { - "id": "10046", - "score": 0.20066462457180023, - "lexicalRank": 293 - }, - { - "id": "10221", - "score": 0.1334114372730255, - "lexicalRank": 258 - }, - { - "id": "10146", - "score": -0.015001742169260979, - "lexicalRank": 273 - }, - { - "id": "10051", - "score": -0.03679070249199867, - "lexicalRank": 292 - }, - { - "id": "10201", - "score": -0.08005551248788834, - "lexicalRank": 262 - }, - { - "id": "10151", - "score": -0.10989139974117279, - "lexicalRank": 272 - }, - { - "id": "10136", - "score": -0.13825947046279907, - "lexicalRank": 275 - }, - { - "id": "10111", - "score": -0.1633528172969818, - "lexicalRank": 280 - } - ], - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "aurora-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10002", - "score": -0.025841891765594482, - "lexicalRank": 121 - }, - { - "id": "10216", - "score": -3.283686399459839, - "lexicalRank": 259 - }, - { - "id": "10141", - "score": -3.294750928878784, - "lexicalRank": 274 - }, - { - "id": "10176", - "score": -3.396474838256836, - "lexicalRank": 267 - }, - { - "id": "10221", - "score": -3.5081534385681152, - "lexicalRank": 258 - }, - { - "id": "10226", - "score": -3.5380752086639404, - "lexicalRank": 257 - }, - { - "id": "10111", - "score": -3.5499141216278076, - "lexicalRank": 280 - }, - { - "id": "10201", - "score": -3.5694005489349365, - "lexicalRank": 262 - }, - { - "id": "10031", - "score": -3.581321954727173, - "lexicalRank": 296 - }, - { - "id": "10086", - "score": -3.582766532897949, - "lexicalRank": 285 - } - ], - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "cedar-question-decision", - "category": "decision", - "relevant": { - "10303": 2, - "10304": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10304", - "score": 5.710914134979248, - "lexicalRank": 121 - }, - { - "id": "10303", - "score": 5.477982044219971, - "lexicalRank": 60 - }, - { - "id": "10524", - "score": 5.470695495605469, - "lexicalRank": 198 - }, - { - "id": "10539", - "score": 5.3812785148620605, - "lexicalRank": 195 - }, - { - "id": "10309", - "score": 5.204921245574951, - "lexicalRank": 241 - }, - { - "id": "10464", - "score": 5.174740314483643, - "lexicalRank": 210 - }, - { - "id": "10454", - "score": 5.0796356201171875, - "lexicalRank": 212 - }, - { - "id": "10494", - "score": 5.079585552215576, - "lexicalRank": 204 - }, - { - "id": "10509", - "score": 5.048130989074707, - "lexicalRank": 201 - }, - { - "id": "10559", - "score": 5.0241827964782715, - "lexicalRank": 191 - } - ], - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "cedar-question-question", - "category": "question", - "relevant": { - "10303": 2, - "10304": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10303", - "score": 10.953361511230469, - "lexicalRank": 60 - }, - { - "id": "10365", - "score": -4.5399861335754395, - "lexicalRank": 47 - }, - { - "id": "10500", - "score": -4.553804874420166, - "lexicalRank": 20 - }, - { - "id": "10450", - "score": -4.634690284729004, - "lexicalRank": 30 - }, - { - "id": "10355", - "score": -4.663148403167725, - "lexicalRank": 49 - }, - { - "id": "10395", - "score": -4.685377597808838, - "lexicalRank": 41 - }, - { - "id": "10340", - "score": -4.763490200042725, - "lexicalRank": 52 - }, - { - "id": "10304", - "score": -4.767633438110352, - "lexicalRank": 121 - }, - { - "id": "10590", - "score": -4.7967634201049805, - "lexicalRank": 2 - }, - { - "id": "10465", - "score": -4.799666881561279, - "lexicalRank": 27 - } - ], - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "cedar-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10304", - "score": 0.1637496054172516, - "lexicalRank": 121 - }, - { - "id": "10468", - "score": -3.1079888343811035, - "lexicalRank": 269 - }, - { - "id": "10413", - "score": -3.2417991161346436, - "lexicalRank": 280 - }, - { - "id": "10348", - "score": -3.2629144191741943, - "lexicalRank": 293 - }, - { - "id": "10593", - "score": -3.302618980407715, - "lexicalRank": 244 - }, - { - "id": "10433", - "score": -3.3093817234039307, - "lexicalRank": 276 - }, - { - "id": "10583", - "score": -3.3447654247283936, - "lexicalRank": 246 - }, - { - "id": "10499", - "score": -3.363835573196411, - "lexicalRank": 203 - }, - { - "id": "10509", - "score": -3.382288694381714, - "lexicalRank": 201 - }, - { - "id": "10488", - "score": -3.3837873935699463, - "lexicalRank": 265 - } - ], - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "север-question-decision", - "category": "decision", - "relevant": { - "10605": 2, - "10606": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10721", - "score": 4.2955641746521, - "lexicalRank": 219 - }, - { - "id": "10796", - "score": 4.140763759613037, - "lexicalRank": 204 - }, - { - "id": "10776", - "score": 4.033090114593506, - "lexicalRank": 208 - }, - { - "id": "10716", - "score": 3.98829984664917, - "lexicalRank": 220 - }, - { - "id": "10841", - "score": 3.8905200958251953, - "lexicalRank": 195 - }, - { - "id": "10696", - "score": 3.8901102542877197, - "lexicalRank": 224 - }, - { - "id": "10866", - "score": 3.8856360912323, - "lexicalRank": 190 - }, - { - "id": "10741", - "score": 3.87813663482666, - "lexicalRank": 215 - }, - { - "id": "10711", - "score": 3.8500914573669434, - "lexicalRank": 221 - }, - { - "id": "10671", - "score": 3.797639846801758, - "lexicalRank": 229 - } - ], - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "север-question-question", - "category": "question", - "relevant": { - "10605": 2, - "10606": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10605", - "score": 6.176127910614014, - "lexicalRank": 1 - }, - { - "id": "10700", - "score": 0.24041175842285156, - "lexicalRank": 283 - }, - { - "id": "10725", - "score": 0.09185468405485153, - "lexicalRank": 278 - }, - { - "id": "10740", - "score": 0.08189026266336441, - "lexicalRank": 275 - }, - { - "id": "10617", - "score": 0.055311575531959534, - "lexicalRank": 59 - }, - { - "id": "10680", - "score": 0.01837293803691864, - "lexicalRank": 287 - }, - { - "id": "10815", - "score": -0.005716301035135984, - "lexicalRank": 260 - }, - { - "id": "10842", - "score": -0.04512234404683113, - "lexicalRank": 14 - }, - { - "id": "10750", - "score": -0.05214595049619675, - "lexicalRank": 273 - }, - { - "id": "10675", - "score": -0.061676062643527985, - "lexicalRank": 288 - } - ], - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "север-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10606", - "score": 1.492608666419983, - "lexicalRank": 0 - }, - { - "id": "10605", - "score": -0.41894200444221497, - "lexicalRank": 1 - }, - { - "id": "10882", - "score": -0.9125255346298218, - "lexicalRank": 6 - }, - { - "id": "10757", - "score": -1.0422241687774658, - "lexicalRank": 31 - }, - { - "id": "10677", - "score": -1.0783100128173828, - "lexicalRank": 47 - }, - { - "id": "10852", - "score": -1.0828702449798584, - "lexicalRank": 12 - }, - { - "id": "10747", - "score": -1.0937329530715942, - "lexicalRank": 33 - }, - { - "id": "10687", - "score": -1.133028268814087, - "lexicalRank": 45 - }, - { - "id": "10637", - "score": -1.160571813583374, - "lexicalRank": 55 - }, - { - "id": "10832", - "score": -1.1662486791610718, - "lexicalRank": 16 - } - ], - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "lumen-question-decision", - "category": "decision", - "relevant": { - "10907": 2, - "10908": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10907", - "score": 10.767654418945312, - "lexicalRank": 61 - }, - { - "id": "10908", - "score": 6.997429370880127, - "lexicalRank": 60 - }, - { - "id": "11094", - "score": 2.228569269180298, - "lexicalRank": 22 - }, - { - "id": "11144", - "score": 2.126849412918091, - "lexicalRank": 12 - }, - { - "id": "11138", - "score": 2.1206576824188232, - "lexicalRank": 196 - }, - { - "id": "11008", - "score": 2.09920597076416, - "lexicalRank": 222 - }, - { - "id": "10978", - "score": 2.0736210346221924, - "lexicalRank": 228 - }, - { - "id": "10948", - "score": 2.0597970485687256, - "lexicalRank": 234 - }, - { - "id": "11118", - "score": 2.0094153881073, - "lexicalRank": 200 - }, - { - "id": "10969", - "score": 1.9855324029922485, - "lexicalRank": 47 - } - ], - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "lumen-question-question", - "category": "question", - "relevant": { - "10907": 2, - "10908": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10907", - "score": 10.034287452697754, - "lexicalRank": 61 - }, - { - "id": "11132", - "score": -0.9715840816497803, - "lexicalRank": 257 - }, - { - "id": "11012", - "score": -1.0529500246047974, - "lexicalRank": 281 - }, - { - "id": "11072", - "score": -1.053822636604309, - "lexicalRank": 269 - }, - { - "id": "11152", - "score": -1.0734130144119263, - "lexicalRank": 253 - }, - { - "id": "11077", - "score": -1.095531702041626, - "lexicalRank": 268 - }, - { - "id": "11157", - "score": -1.1130152940750122, - "lexicalRank": 252 - }, - { - "id": "11107", - "score": -1.1273714303970337, - "lexicalRank": 262 - }, - { - "id": "11127", - "score": -1.1431617736816406, - "lexicalRank": 258 - }, - { - "id": "11042", - "score": -1.1596165895462036, - "lexicalRank": 275 - } - ], - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "lumen-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10908", - "score": 0.5671241879463196, - "lexicalRank": 60 - }, - { - "id": "11192", - "score": -2.253211259841919, - "lexicalRank": 245 - }, - { - "id": "11107", - "score": -2.3072736263275146, - "lexicalRank": 262 - }, - { - "id": "10907", - "score": -2.4978764057159424, - "lexicalRank": 61 - }, - { - "id": "11092", - "score": -2.517425537109375, - "lexicalRank": 265 - }, - { - "id": "10997", - "score": -2.5706686973571777, - "lexicalRank": 284 - }, - { - "id": "10967", - "score": -2.5796711444854736, - "lexicalRank": 290 - }, - { - "id": "11142", - "score": -2.6291210651397705, - "lexicalRank": 255 - }, - { - "id": "11022", - "score": -2.6314783096313477, - "lexicalRank": 279 - }, - { - "id": "11137", - "score": -2.6895358562469482, - "lexicalRank": 256 - } - ], - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ] - } - ] -} diff --git a/bench/message-search-quality/crossencoder.ts b/bench/message-search-quality/crossencoder.ts deleted file mode 100644 index 3d2201fdb..000000000 --- a/bench/message-search-quality/crossencoder.ts +++ /dev/null @@ -1,66 +0,0 @@ -import assert from "node:assert/strict" -import { createHash } from "node:crypto" -import { readFileSync } from "node:fs" -import { join } from "node:path" - -export const openCrossEncoder = async (directory: string) => { - const manifest = JSON.parse(readFileSync(new URL("./reranker.json", import.meta.url), "utf8")) as { - repository: string - revision: string - licence: string - maxTokens: number - files: { name: string; sha256: string; url: string }[] - } - for (const file of manifest.files) - assert.equal( - createHash("sha256") - .update(readFileSync(join(directory, file.name))) - .digest("hex"), - file.sha256, - `pinned reranker ${file.name}`, - ) - const config = JSON.parse(readFileSync(join(directory, "config.json"), "utf8")) as { - sbert_ce_default_activation_function: string - id2label: Record - } - assert.equal(config.sbert_ce_default_activation_function, "torch.nn.modules.linear.Identity") - assert.equal(Object.keys(config.id2label).length, 1) - const [{ Tokenizer }, ort] = await Promise.all([ - import("@huggingface/tokenizers"), - import("@leemour/cli-messaging-onnx"), - ]) - const tokenizer = new Tokenizer( - JSON.parse(readFileSync(join(directory, "tokenizer.json"), "utf8")), - JSON.parse(readFileSync(join(directory, "tokenizer_config.json"), "utf8")), - ) - ort.env.wasm.numThreads = 8 - const model = manifest.files.find((file) => file.name.endsWith(".onnx")) - assert.ok(model) - const session = await ort.InferenceSession.create(readFileSync(join(directory, model.name))) - assert.deepEqual([...session.inputNames].sort(), ["attention_mask", "input_ids"]) - assert.deepEqual(session.outputNames, ["logits"]) - return { - model: { - id: manifest.repository, - revision: manifest.revision, - licence: manifest.licence, - files: manifest.files.map(({ name, sha256 }) => ({ name, sha256 })), - threads: 8, - }, - score: async (query: string, passage: string) => { - const ids = tokenizer.encode(query, { text_pair: passage }).ids - assert.ok(ids.length <= manifest.maxTokens, "benchmark pair must fit without truncation") - const result = await session.run({ - input_ids: new ort.Tensor("int64", BigInt64Array.from(ids, BigInt), [1, ids.length]), - attention_mask: new ort.Tensor("int64", new BigInt64Array(ids.length).fill(1n), [1, ids.length]), - }) - const logits = result.logits - assert.ok(logits) - assert.equal(logits.data.length, 1) - const score = Number(logits.data[0]) - assert.ok(Number.isFinite(score)) - return score - }, - close: () => session.release(), - } -} diff --git a/bench/message-search-quality/generate-questions.py b/bench/message-search-quality/generate-questions.py deleted file mode 100644 index 3d8be606d..000000000 --- a/bench/message-search-quality/generate-questions.py +++ /dev/null @@ -1,50 +0,0 @@ -import json -from pathlib import Path - -root = Path(__file__).parent -original = json.loads((root / 'corpus.json').read_text()) -queries, chats, messages = [], [], [] - -# Questions state the task, never the labelled answer. Original keyword queries remain unchanged. -questions = [ - ('aurora', 'aurora rollout rollback', 'What rollback decision was agreed for the Aurora rollout?', 'What should we do to roll back the Aurora rollout?'), - ('cedar', 'cedar backup retention', 'What retention policy was agreed for Cedar backups?', 'How long should Cedar backup snapshots be kept?'), - ('север', 'север сертификат обновление', 'Какое решение принято по обновлению сертификата Север?', 'Что известно о новом сертификате Север и сроке его действия?'), - ('lumen', 'lumen invoice refund', 'What refund was approved for the Lumen invoice?', 'What should be returned to the payer of the Lumen invoice?'), -] -for family, anchors, decision, question in questions: - relevant = next(q['relevant'] for q in original['queries'] if q['id'] == f'{family}-words') - for category, text in [('decision', decision), ('question', question)]: - queries.append(dict(id=f'{family}-question-{category}', split='dev', family=family, category=category, retrieval=anchors, text=text, relevant=relevant)) - text = f'Which independent auditor signed off on {anchors}?' if family != 'север' else 'Какой независимый аудитор подписал обновление сертификата Север?' - queries.append(dict(id=f'{family}-question-no-answer', split='dev', family=family, category='no-answer', retrieval=anchors, text=text, relevant={})) - -fresh = [ - ('cobalt', 'cobalt failover routing', 'Cobalt failover routing approved: route traffic to the west pool and hold the east pool offline.', 'Cobalt failover routing owner is Lea; the approved change is documented.', 'What failover routing was approved for Cobalt?', 'Where should Cobalt traffic go after failover?'), - ('birch', 'birch export encryption', 'Birch export encryption policy: encrypt every export with a per-recipient key before sending it.', 'Birch export encryption owner is Arun; the agreed policy is documented.', 'What export encryption policy was agreed for Birch?', 'How should Birch exports be encrypted before they are sent?'), - ('спектр', 'спектр очередь лимит', 'Спектр очередь лимит согласован: принимать не более 120 задач, остальные отправлять в резервную очередь.', 'Спектр очередь лимит подтвержден, владелец Даша.', 'Какой лимит очереди согласован для Спектр?', 'Сколько задач принимает очередь Спектр и куда отправлять остальные?'), - ('nova', 'nova access expiry', 'Nova access expiry policy: expire temporary access after 48 hours and require a new approval to extend it.', 'Nova access expiry owner is Bo; the agreed policy is documented.', 'What access expiry policy was agreed for Nova?', 'When does Nova temporary access expire and how can it be extended?'), -] -mid = 70000 -for i, (family, anchors, answer, support, decision, question) in enumerate(fresh, 201): - cid = str(i) - chats.append(dict(id=cid, title=f'Synthetic {family} decisions')) - relevant = {} - for grade, text in [(2, answer), (1, support)]: - mid += 1 - messages.append(dict(id=str(mid), chatId=cid, senderId='800', timestamp='2026-10-05T12:00:00.000Z', text=text)) - relevant[str(mid)] = grade - for j in range(150): - text = [f'{anchors}: put this item on the agenda. No agreement was reached.', - f'{anchors}: rejected suggestion copied from a draft, awaiting a decision.', - f'{anchors}: empty form awaiting the actual policy.', - f'Checking whether the team discussed {anchors}. This is a question, not a confirmed decision.', - f'{family} keyword list: {anchors}; no instruction is recorded.'][j % 5] - mid += 1 - messages.append(dict(id=str(mid), chatId=cid, senderId='801', timestamp='2026-10-06T12:00:00.000Z', text=f'{text} Item {j}.')) - for category, text in [('decision', decision), ('question', question)]: - queries.append(dict(id=f'{family}-question-{category}', split='test', family=family, category=category, retrieval=anchors, text=text, relevant=dict(relevant))) - text = f'Which independent auditor signed off on {anchors}?' if family != 'спектр' else 'Какой независимый аудитор подписал лимит очереди Спектр?' - queries.append(dict(id=f'{family}-question-no-answer', split='test', family=family, category='no-answer', retrieval=anchors, text=text, relevant={})) - -(root / 'questions.json').write_text(json.dumps(dict(chats=chats, messages=messages, queries=queries), ensure_ascii=False, indent=2) + '\n') diff --git a/bench/message-search-quality/generate.py b/bench/message-search-quality/generate.py deleted file mode 100644 index 12a7b26e9..000000000 --- a/bench/message-search-quality/generate.py +++ /dev/null @@ -1,60 +0,0 @@ -import json -from pathlib import Path - -# Fixed invented topics; the emitted corpus, rather than a random seed, is the evaluation input. -families = [ - ('dev', 'aurora', 'rollout', 'rollback', 'Aurora rollout rollback decision: disable the canary and restore build 17.', 'The Aurora rollout rollback owner is Mira; the decision is recorded above.', 'rolluot', 'roll', 'deployment reversal'), - ('dev', 'cedar', 'backup', 'retention', 'Cedar backup retention decision: keep daily snapshots for 35 days.', 'The Cedar backup retention owner is Niko; the policy is recorded above.', 'bakcup', 'back', 'snapshot lifetime'), - ('dev', 'север', 'сертификат', 'обновление', 'Север сертификат обновление: новый сертификат установлен, срок до 12 декабря.', 'Север сертификат обновление проверено, владелец Ира.', 'сертифкат', 'серти', 'продление TLS'), - ('dev', 'lumen', 'invoice', 'refund', 'Lumen invoice refund approved: return 480 credits to the original payer.', 'The Lumen invoice refund owner is Eva; approval is recorded above.', 'invocie', 'invo', 'payment reimbursement'), - ('test', 'harbor', 'release', 'freeze', 'Harbor release freeze decision: stop deployments until the database check passes.', 'The Harbor release freeze owner is Oren; the decision is recorded above.', 'relesae', 'rele', 'shipping pause'), - ('test', 'maple', 'archive', 'restore', 'Maple archive restore decision: recover the March snapshot into a separate workspace.', 'The Maple archive restore owner is Ada; the decision is recorded above.', 'arhcive', 'arch', 'recover saved data'), - ('test', 'вектор', 'миграция', 'отмена', 'Вектор миграция отмена: отключить задание и вернуть схему версии 8.', 'Вектор миграция отмена проверена, владелец Лев.', 'миграця', 'мигр', 'возврат прежней схемы'), - ('test', 'prism', 'renewal', 'discount', 'Prism renewal discount approved: apply 15 percent to the annual contract.', 'The Prism renewal discount owner is Tess; approval is recorded above.', 'renweal', 'rene', 'subscription price reduction'), -] -messages, queries = [], [] -chats = [{'id': str(i), 'title': f'Synthetic {name} decisions'} for i, (_, name, *_) in enumerate(families, 101)] -chats.append({'id': '999', 'title': 'Synthetic busy operations'}) -next_id = 10000 - -def add(chat, sender, text, day=2): - global next_id - next_id += 1 - mid = str(next_id) - messages.append(dict(id=mid, chatId=chat, senderId=sender, timestamp=f'2026-10-{day:02d}T12:00:00.000Z', text=text)) - return mid - -for i, (split, name, word, action, answer, support, typo, prefix, meaning) in enumerate(families, 101): - chat = str(i) - a = add(chat, '700', answer) - b = add(chat, '701', support) - for j in range(300): - templates = [ - f'{name} {word} {action}: agenda only; no decision yet. Thread {j}.', - f'{name} {word} {action}? Please add this question to the meeting. Thread {j}.', - f'Checklist heading: {name} {word} {action}. This empty template supplies no policy. Thread {j}.', - f'{name} status digest. {word} is a document tag. {action} is a separate training exercise. No answer. Thread {j}.', - f'Old {name} {word} {action} proposal rejected; do not treat this as the agreed decision. Thread {j}.', - ] - add('999' if j < 200 else chat, '702', templates[j % len(templates)], 3) - variants = [ - ('words', f'{name} {word} {action}', None), - ('typo', f'{name} {typo} {action}', None), - ('beginnings', f'{name} {prefix} {action}', None), - ('phrase', f'"{name} {word} {action}"', None), - ('from', f'{name} {word} from:700', None), - ('chat', f'{name} {word} chat:{chat}', None), - ('date', f'{name} {word} date:2026-10-02', f'{name} {word} after:2026-10-02 before:2026-10-03'), - ('and', f'{name} AND {word} AND {action}', f'{name} {word} {action}'), - ('or', f'{name} {word} OR {action}', None), - ('paraphrase', f'{name} {meaning}', None), - ] - for category, text, legacy in variants: - relevant = {a: 2, b: 1} - if category == 'from': relevant = {a: 2} - queries.append(dict(id=f'{name}-{category}', split=split, category=category, intent=answer, lucene=text, legacy=legacy or text, relevant=relevant)) - for category, text in [('absent', f'{name} quantum unicorn'), ('near-miss', f'{name} {word} {action} purple')]: - queries.append(dict(id=f'{name}-{category}', split=split, category=f'no-answer-{category}', intent='No message answers the requested combination.', lucene=text, legacy=text, relevant={})) - -for j in range(200): add('999', '703', f'Unrelated lunch timetable and office stationery reminder {j}.', 4) -Path(__file__).with_name('corpus.json').write_text(json.dumps(dict(chats=chats, messages=messages, queries=queries), ensure_ascii=False, indent=2) + '\n') diff --git a/bench/message-search-quality/prepare-reranker.py b/bench/message-search-quality/prepare-reranker.py deleted file mode 100644 index b20643a69..000000000 --- a/bench/message-search-quality/prepare-reranker.py +++ /dev/null @@ -1,25 +0,0 @@ -"""Explicit model preparation, kept separate from the offline benchmark.""" -import hashlib -import json -import pathlib -import sys -import urllib.request - -root = pathlib.Path(sys.argv[1]) -manifest = json.loads(pathlib.Path(__file__).with_name('reranker.json').read_text()) -root.mkdir(parents=True, exist_ok=True) -for file in manifest['files']: - target = root / file['name'] - target.parent.mkdir(parents=True, exist_ok=True) - if not target.exists(): - staged = target.with_name(target.name + '.partial') - print('Preparing pinned model file:', file['name'], flush=True) - with urllib.request.urlopen(file['url']) as response, staged.open('wb') as output: - while block := response.read(1024 * 1024): - output.write(block) - actual = hashlib.sha256(staged.read_bytes()).hexdigest() - if actual != file['sha256']: - raise ValueError(f"Hash mismatch for {file['name']}; partial file retained for inspection") - staged.rename(target) - assert hashlib.sha256(target.read_bytes()).hexdigest() == file['sha256'], file['name'] -print('Verified local model directory:', root) diff --git a/bench/message-search-quality/questions.json b/bench/message-search-quality/questions.json deleted file mode 100644 index c075065fa..000000000 --- a/bench/message-search-quality/questions.json +++ /dev/null @@ -1,4544 +0,0 @@ -{ - "chats": [ - { - "id": "201", - "title": "Synthetic cobalt decisions" - }, - { - "id": "202", - "title": "Synthetic birch decisions" - }, - { - "id": "203", - "title": "Synthetic спектр decisions" - }, - { - "id": "204", - "title": "Synthetic nova decisions" - } - ], - "messages": [ - { - "id": "70001", - "chatId": "201", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Cobalt failover routing approved: route traffic to the west pool and hold the east pool offline." - }, - { - "id": "70002", - "chatId": "201", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Cobalt failover routing owner is Lea; the approved change is documented." - }, - { - "id": "70003", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 0." - }, - { - "id": "70004", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 1." - }, - { - "id": "70005", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 2." - }, - { - "id": "70006", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 3." - }, - { - "id": "70007", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 4." - }, - { - "id": "70008", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 5." - }, - { - "id": "70009", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 6." - }, - { - "id": "70010", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 7." - }, - { - "id": "70011", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 8." - }, - { - "id": "70012", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 9." - }, - { - "id": "70013", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 10." - }, - { - "id": "70014", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 11." - }, - { - "id": "70015", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 12." - }, - { - "id": "70016", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 13." - }, - { - "id": "70017", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 14." - }, - { - "id": "70018", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 15." - }, - { - "id": "70019", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 16." - }, - { - "id": "70020", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 17." - }, - { - "id": "70021", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 18." - }, - { - "id": "70022", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 19." - }, - { - "id": "70023", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 20." - }, - { - "id": "70024", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 21." - }, - { - "id": "70025", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 22." - }, - { - "id": "70026", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 23." - }, - { - "id": "70027", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 24." - }, - { - "id": "70028", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 25." - }, - { - "id": "70029", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 26." - }, - { - "id": "70030", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 27." - }, - { - "id": "70031", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 28." - }, - { - "id": "70032", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 29." - }, - { - "id": "70033", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 30." - }, - { - "id": "70034", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 31." - }, - { - "id": "70035", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 32." - }, - { - "id": "70036", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 33." - }, - { - "id": "70037", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 34." - }, - { - "id": "70038", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 35." - }, - { - "id": "70039", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 36." - }, - { - "id": "70040", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 37." - }, - { - "id": "70041", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 38." - }, - { - "id": "70042", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 39." - }, - { - "id": "70043", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 40." - }, - { - "id": "70044", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 41." - }, - { - "id": "70045", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 42." - }, - { - "id": "70046", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 43." - }, - { - "id": "70047", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 44." - }, - { - "id": "70048", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 45." - }, - { - "id": "70049", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 46." - }, - { - "id": "70050", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 47." - }, - { - "id": "70051", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 48." - }, - { - "id": "70052", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 49." - }, - { - "id": "70053", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 50." - }, - { - "id": "70054", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 51." - }, - { - "id": "70055", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 52." - }, - { - "id": "70056", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 53." - }, - { - "id": "70057", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 54." - }, - { - "id": "70058", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 55." - }, - { - "id": "70059", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 56." - }, - { - "id": "70060", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 57." - }, - { - "id": "70061", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 58." - }, - { - "id": "70062", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 59." - }, - { - "id": "70063", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 60." - }, - { - "id": "70064", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 61." - }, - { - "id": "70065", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 62." - }, - { - "id": "70066", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 63." - }, - { - "id": "70067", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 64." - }, - { - "id": "70068", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 65." - }, - { - "id": "70069", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 66." - }, - { - "id": "70070", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 67." - }, - { - "id": "70071", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 68." - }, - { - "id": "70072", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 69." - }, - { - "id": "70073", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 70." - }, - { - "id": "70074", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 71." - }, - { - "id": "70075", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 72." - }, - { - "id": "70076", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 73." - }, - { - "id": "70077", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 74." - }, - { - "id": "70078", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 75." - }, - { - "id": "70079", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 76." - }, - { - "id": "70080", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 77." - }, - { - "id": "70081", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 78." - }, - { - "id": "70082", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 79." - }, - { - "id": "70083", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 80." - }, - { - "id": "70084", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 81." - }, - { - "id": "70085", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 82." - }, - { - "id": "70086", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 83." - }, - { - "id": "70087", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 84." - }, - { - "id": "70088", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 85." - }, - { - "id": "70089", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 86." - }, - { - "id": "70090", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 87." - }, - { - "id": "70091", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 88." - }, - { - "id": "70092", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 89." - }, - { - "id": "70093", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 90." - }, - { - "id": "70094", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 91." - }, - { - "id": "70095", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 92." - }, - { - "id": "70096", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 93." - }, - { - "id": "70097", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 94." - }, - { - "id": "70098", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 95." - }, - { - "id": "70099", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 96." - }, - { - "id": "70100", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 97." - }, - { - "id": "70101", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 98." - }, - { - "id": "70102", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 99." - }, - { - "id": "70103", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 100." - }, - { - "id": "70104", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 101." - }, - { - "id": "70105", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 102." - }, - { - "id": "70106", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 103." - }, - { - "id": "70107", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 104." - }, - { - "id": "70108", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 105." - }, - { - "id": "70109", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 106." - }, - { - "id": "70110", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 107." - }, - { - "id": "70111", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 108." - }, - { - "id": "70112", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 109." - }, - { - "id": "70113", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 110." - }, - { - "id": "70114", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 111." - }, - { - "id": "70115", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 112." - }, - { - "id": "70116", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 113." - }, - { - "id": "70117", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 114." - }, - { - "id": "70118", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 115." - }, - { - "id": "70119", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 116." - }, - { - "id": "70120", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 117." - }, - { - "id": "70121", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 118." - }, - { - "id": "70122", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 119." - }, - { - "id": "70123", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 120." - }, - { - "id": "70124", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 121." - }, - { - "id": "70125", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 122." - }, - { - "id": "70126", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 123." - }, - { - "id": "70127", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 124." - }, - { - "id": "70128", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 125." - }, - { - "id": "70129", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 126." - }, - { - "id": "70130", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 127." - }, - { - "id": "70131", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 128." - }, - { - "id": "70132", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 129." - }, - { - "id": "70133", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 130." - }, - { - "id": "70134", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 131." - }, - { - "id": "70135", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 132." - }, - { - "id": "70136", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 133." - }, - { - "id": "70137", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 134." - }, - { - "id": "70138", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 135." - }, - { - "id": "70139", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 136." - }, - { - "id": "70140", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 137." - }, - { - "id": "70141", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 138." - }, - { - "id": "70142", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 139." - }, - { - "id": "70143", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 140." - }, - { - "id": "70144", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 141." - }, - { - "id": "70145", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 142." - }, - { - "id": "70146", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 143." - }, - { - "id": "70147", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 144." - }, - { - "id": "70148", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: put this item on the agenda. No agreement was reached. Item 145." - }, - { - "id": "70149", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: rejected suggestion copied from a draft, awaiting a decision. Item 146." - }, - { - "id": "70150", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt failover routing: empty form awaiting the actual policy. Item 147." - }, - { - "id": "70151", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed cobalt failover routing. This is a question, not a confirmed decision. Item 148." - }, - { - "id": "70152", - "chatId": "201", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "cobalt keyword list: cobalt failover routing; no instruction is recorded. Item 149." - }, - { - "id": "70153", - "chatId": "202", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Birch export encryption policy: encrypt every export with a per-recipient key before sending it." - }, - { - "id": "70154", - "chatId": "202", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Birch export encryption owner is Arun; the agreed policy is documented." - }, - { - "id": "70155", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 0." - }, - { - "id": "70156", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 1." - }, - { - "id": "70157", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 2." - }, - { - "id": "70158", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 3." - }, - { - "id": "70159", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 4." - }, - { - "id": "70160", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 5." - }, - { - "id": "70161", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 6." - }, - { - "id": "70162", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 7." - }, - { - "id": "70163", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 8." - }, - { - "id": "70164", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 9." - }, - { - "id": "70165", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 10." - }, - { - "id": "70166", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 11." - }, - { - "id": "70167", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 12." - }, - { - "id": "70168", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 13." - }, - { - "id": "70169", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 14." - }, - { - "id": "70170", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 15." - }, - { - "id": "70171", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 16." - }, - { - "id": "70172", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 17." - }, - { - "id": "70173", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 18." - }, - { - "id": "70174", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 19." - }, - { - "id": "70175", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 20." - }, - { - "id": "70176", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 21." - }, - { - "id": "70177", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 22." - }, - { - "id": "70178", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 23." - }, - { - "id": "70179", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 24." - }, - { - "id": "70180", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 25." - }, - { - "id": "70181", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 26." - }, - { - "id": "70182", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 27." - }, - { - "id": "70183", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 28." - }, - { - "id": "70184", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 29." - }, - { - "id": "70185", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 30." - }, - { - "id": "70186", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 31." - }, - { - "id": "70187", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 32." - }, - { - "id": "70188", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 33." - }, - { - "id": "70189", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 34." - }, - { - "id": "70190", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 35." - }, - { - "id": "70191", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 36." - }, - { - "id": "70192", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 37." - }, - { - "id": "70193", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 38." - }, - { - "id": "70194", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 39." - }, - { - "id": "70195", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 40." - }, - { - "id": "70196", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 41." - }, - { - "id": "70197", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 42." - }, - { - "id": "70198", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 43." - }, - { - "id": "70199", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 44." - }, - { - "id": "70200", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 45." - }, - { - "id": "70201", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 46." - }, - { - "id": "70202", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 47." - }, - { - "id": "70203", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 48." - }, - { - "id": "70204", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 49." - }, - { - "id": "70205", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 50." - }, - { - "id": "70206", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 51." - }, - { - "id": "70207", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 52." - }, - { - "id": "70208", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 53." - }, - { - "id": "70209", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 54." - }, - { - "id": "70210", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 55." - }, - { - "id": "70211", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 56." - }, - { - "id": "70212", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 57." - }, - { - "id": "70213", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 58." - }, - { - "id": "70214", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 59." - }, - { - "id": "70215", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 60." - }, - { - "id": "70216", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 61." - }, - { - "id": "70217", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 62." - }, - { - "id": "70218", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 63." - }, - { - "id": "70219", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 64." - }, - { - "id": "70220", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 65." - }, - { - "id": "70221", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 66." - }, - { - "id": "70222", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 67." - }, - { - "id": "70223", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 68." - }, - { - "id": "70224", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 69." - }, - { - "id": "70225", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 70." - }, - { - "id": "70226", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 71." - }, - { - "id": "70227", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 72." - }, - { - "id": "70228", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 73." - }, - { - "id": "70229", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 74." - }, - { - "id": "70230", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 75." - }, - { - "id": "70231", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 76." - }, - { - "id": "70232", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 77." - }, - { - "id": "70233", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 78." - }, - { - "id": "70234", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 79." - }, - { - "id": "70235", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 80." - }, - { - "id": "70236", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 81." - }, - { - "id": "70237", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 82." - }, - { - "id": "70238", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 83." - }, - { - "id": "70239", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 84." - }, - { - "id": "70240", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 85." - }, - { - "id": "70241", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 86." - }, - { - "id": "70242", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 87." - }, - { - "id": "70243", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 88." - }, - { - "id": "70244", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 89." - }, - { - "id": "70245", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 90." - }, - { - "id": "70246", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 91." - }, - { - "id": "70247", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 92." - }, - { - "id": "70248", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 93." - }, - { - "id": "70249", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 94." - }, - { - "id": "70250", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 95." - }, - { - "id": "70251", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 96." - }, - { - "id": "70252", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 97." - }, - { - "id": "70253", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 98." - }, - { - "id": "70254", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 99." - }, - { - "id": "70255", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 100." - }, - { - "id": "70256", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 101." - }, - { - "id": "70257", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 102." - }, - { - "id": "70258", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 103." - }, - { - "id": "70259", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 104." - }, - { - "id": "70260", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 105." - }, - { - "id": "70261", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 106." - }, - { - "id": "70262", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 107." - }, - { - "id": "70263", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 108." - }, - { - "id": "70264", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 109." - }, - { - "id": "70265", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 110." - }, - { - "id": "70266", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 111." - }, - { - "id": "70267", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 112." - }, - { - "id": "70268", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 113." - }, - { - "id": "70269", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 114." - }, - { - "id": "70270", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 115." - }, - { - "id": "70271", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 116." - }, - { - "id": "70272", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 117." - }, - { - "id": "70273", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 118." - }, - { - "id": "70274", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 119." - }, - { - "id": "70275", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 120." - }, - { - "id": "70276", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 121." - }, - { - "id": "70277", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 122." - }, - { - "id": "70278", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 123." - }, - { - "id": "70279", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 124." - }, - { - "id": "70280", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 125." - }, - { - "id": "70281", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 126." - }, - { - "id": "70282", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 127." - }, - { - "id": "70283", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 128." - }, - { - "id": "70284", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 129." - }, - { - "id": "70285", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 130." - }, - { - "id": "70286", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 131." - }, - { - "id": "70287", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 132." - }, - { - "id": "70288", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 133." - }, - { - "id": "70289", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 134." - }, - { - "id": "70290", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 135." - }, - { - "id": "70291", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 136." - }, - { - "id": "70292", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 137." - }, - { - "id": "70293", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 138." - }, - { - "id": "70294", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 139." - }, - { - "id": "70295", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 140." - }, - { - "id": "70296", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 141." - }, - { - "id": "70297", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 142." - }, - { - "id": "70298", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 143." - }, - { - "id": "70299", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 144." - }, - { - "id": "70300", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: put this item on the agenda. No agreement was reached. Item 145." - }, - { - "id": "70301", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: rejected suggestion copied from a draft, awaiting a decision. Item 146." - }, - { - "id": "70302", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch export encryption: empty form awaiting the actual policy. Item 147." - }, - { - "id": "70303", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed birch export encryption. This is a question, not a confirmed decision. Item 148." - }, - { - "id": "70304", - "chatId": "202", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "birch keyword list: birch export encryption; no instruction is recorded. Item 149." - }, - { - "id": "70305", - "chatId": "203", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Спектр очередь лимит согласован: принимать не более 120 задач, остальные отправлять в резервную очередь." - }, - { - "id": "70306", - "chatId": "203", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Спектр очередь лимит подтвержден, владелец Даша." - }, - { - "id": "70307", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 0." - }, - { - "id": "70308", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 1." - }, - { - "id": "70309", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 2." - }, - { - "id": "70310", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 3." - }, - { - "id": "70311", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 4." - }, - { - "id": "70312", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 5." - }, - { - "id": "70313", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 6." - }, - { - "id": "70314", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 7." - }, - { - "id": "70315", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 8." - }, - { - "id": "70316", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 9." - }, - { - "id": "70317", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 10." - }, - { - "id": "70318", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 11." - }, - { - "id": "70319", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 12." - }, - { - "id": "70320", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 13." - }, - { - "id": "70321", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 14." - }, - { - "id": "70322", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 15." - }, - { - "id": "70323", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 16." - }, - { - "id": "70324", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 17." - }, - { - "id": "70325", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 18." - }, - { - "id": "70326", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 19." - }, - { - "id": "70327", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 20." - }, - { - "id": "70328", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 21." - }, - { - "id": "70329", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 22." - }, - { - "id": "70330", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 23." - }, - { - "id": "70331", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 24." - }, - { - "id": "70332", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 25." - }, - { - "id": "70333", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 26." - }, - { - "id": "70334", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 27." - }, - { - "id": "70335", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 28." - }, - { - "id": "70336", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 29." - }, - { - "id": "70337", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 30." - }, - { - "id": "70338", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 31." - }, - { - "id": "70339", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 32." - }, - { - "id": "70340", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 33." - }, - { - "id": "70341", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 34." - }, - { - "id": "70342", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 35." - }, - { - "id": "70343", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 36." - }, - { - "id": "70344", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 37." - }, - { - "id": "70345", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 38." - }, - { - "id": "70346", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 39." - }, - { - "id": "70347", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 40." - }, - { - "id": "70348", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 41." - }, - { - "id": "70349", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 42." - }, - { - "id": "70350", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 43." - }, - { - "id": "70351", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 44." - }, - { - "id": "70352", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 45." - }, - { - "id": "70353", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 46." - }, - { - "id": "70354", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 47." - }, - { - "id": "70355", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 48." - }, - { - "id": "70356", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 49." - }, - { - "id": "70357", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 50." - }, - { - "id": "70358", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 51." - }, - { - "id": "70359", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 52." - }, - { - "id": "70360", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 53." - }, - { - "id": "70361", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 54." - }, - { - "id": "70362", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 55." - }, - { - "id": "70363", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 56." - }, - { - "id": "70364", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 57." - }, - { - "id": "70365", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 58." - }, - { - "id": "70366", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 59." - }, - { - "id": "70367", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 60." - }, - { - "id": "70368", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 61." - }, - { - "id": "70369", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 62." - }, - { - "id": "70370", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 63." - }, - { - "id": "70371", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 64." - }, - { - "id": "70372", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 65." - }, - { - "id": "70373", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 66." - }, - { - "id": "70374", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 67." - }, - { - "id": "70375", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 68." - }, - { - "id": "70376", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 69." - }, - { - "id": "70377", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 70." - }, - { - "id": "70378", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 71." - }, - { - "id": "70379", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 72." - }, - { - "id": "70380", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 73." - }, - { - "id": "70381", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 74." - }, - { - "id": "70382", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 75." - }, - { - "id": "70383", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 76." - }, - { - "id": "70384", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 77." - }, - { - "id": "70385", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 78." - }, - { - "id": "70386", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 79." - }, - { - "id": "70387", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 80." - }, - { - "id": "70388", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 81." - }, - { - "id": "70389", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 82." - }, - { - "id": "70390", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 83." - }, - { - "id": "70391", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 84." - }, - { - "id": "70392", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 85." - }, - { - "id": "70393", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 86." - }, - { - "id": "70394", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 87." - }, - { - "id": "70395", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 88." - }, - { - "id": "70396", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 89." - }, - { - "id": "70397", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 90." - }, - { - "id": "70398", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 91." - }, - { - "id": "70399", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 92." - }, - { - "id": "70400", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 93." - }, - { - "id": "70401", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 94." - }, - { - "id": "70402", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 95." - }, - { - "id": "70403", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 96." - }, - { - "id": "70404", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 97." - }, - { - "id": "70405", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 98." - }, - { - "id": "70406", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 99." - }, - { - "id": "70407", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 100." - }, - { - "id": "70408", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 101." - }, - { - "id": "70409", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 102." - }, - { - "id": "70410", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 103." - }, - { - "id": "70411", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 104." - }, - { - "id": "70412", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 105." - }, - { - "id": "70413", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 106." - }, - { - "id": "70414", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 107." - }, - { - "id": "70415", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 108." - }, - { - "id": "70416", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 109." - }, - { - "id": "70417", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 110." - }, - { - "id": "70418", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 111." - }, - { - "id": "70419", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 112." - }, - { - "id": "70420", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 113." - }, - { - "id": "70421", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 114." - }, - { - "id": "70422", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 115." - }, - { - "id": "70423", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 116." - }, - { - "id": "70424", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 117." - }, - { - "id": "70425", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 118." - }, - { - "id": "70426", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 119." - }, - { - "id": "70427", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 120." - }, - { - "id": "70428", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 121." - }, - { - "id": "70429", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 122." - }, - { - "id": "70430", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 123." - }, - { - "id": "70431", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 124." - }, - { - "id": "70432", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 125." - }, - { - "id": "70433", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 126." - }, - { - "id": "70434", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 127." - }, - { - "id": "70435", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 128." - }, - { - "id": "70436", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 129." - }, - { - "id": "70437", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 130." - }, - { - "id": "70438", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 131." - }, - { - "id": "70439", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 132." - }, - { - "id": "70440", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 133." - }, - { - "id": "70441", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 134." - }, - { - "id": "70442", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 135." - }, - { - "id": "70443", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 136." - }, - { - "id": "70444", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 137." - }, - { - "id": "70445", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 138." - }, - { - "id": "70446", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 139." - }, - { - "id": "70447", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 140." - }, - { - "id": "70448", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 141." - }, - { - "id": "70449", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 142." - }, - { - "id": "70450", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 143." - }, - { - "id": "70451", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 144." - }, - { - "id": "70452", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: put this item on the agenda. No agreement was reached. Item 145." - }, - { - "id": "70453", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: rejected suggestion copied from a draft, awaiting a decision. Item 146." - }, - { - "id": "70454", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр очередь лимит: empty form awaiting the actual policy. Item 147." - }, - { - "id": "70455", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed спектр очередь лимит. This is a question, not a confirmed decision. Item 148." - }, - { - "id": "70456", - "chatId": "203", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "спектр keyword list: спектр очередь лимит; no instruction is recorded. Item 149." - }, - { - "id": "70457", - "chatId": "204", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Nova access expiry policy: expire temporary access after 48 hours and require a new approval to extend it." - }, - { - "id": "70458", - "chatId": "204", - "senderId": "800", - "timestamp": "2026-10-05T12:00:00.000Z", - "text": "Nova access expiry owner is Bo; the agreed policy is documented." - }, - { - "id": "70459", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 0." - }, - { - "id": "70460", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 1." - }, - { - "id": "70461", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 2." - }, - { - "id": "70462", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 3." - }, - { - "id": "70463", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 4." - }, - { - "id": "70464", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 5." - }, - { - "id": "70465", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 6." - }, - { - "id": "70466", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 7." - }, - { - "id": "70467", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 8." - }, - { - "id": "70468", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 9." - }, - { - "id": "70469", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 10." - }, - { - "id": "70470", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 11." - }, - { - "id": "70471", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 12." - }, - { - "id": "70472", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 13." - }, - { - "id": "70473", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 14." - }, - { - "id": "70474", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 15." - }, - { - "id": "70475", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 16." - }, - { - "id": "70476", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 17." - }, - { - "id": "70477", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 18." - }, - { - "id": "70478", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 19." - }, - { - "id": "70479", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 20." - }, - { - "id": "70480", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 21." - }, - { - "id": "70481", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 22." - }, - { - "id": "70482", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 23." - }, - { - "id": "70483", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 24." - }, - { - "id": "70484", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 25." - }, - { - "id": "70485", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 26." - }, - { - "id": "70486", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 27." - }, - { - "id": "70487", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 28." - }, - { - "id": "70488", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 29." - }, - { - "id": "70489", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 30." - }, - { - "id": "70490", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 31." - }, - { - "id": "70491", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 32." - }, - { - "id": "70492", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 33." - }, - { - "id": "70493", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 34." - }, - { - "id": "70494", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 35." - }, - { - "id": "70495", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 36." - }, - { - "id": "70496", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 37." - }, - { - "id": "70497", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 38." - }, - { - "id": "70498", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 39." - }, - { - "id": "70499", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 40." - }, - { - "id": "70500", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 41." - }, - { - "id": "70501", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 42." - }, - { - "id": "70502", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 43." - }, - { - "id": "70503", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 44." - }, - { - "id": "70504", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 45." - }, - { - "id": "70505", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 46." - }, - { - "id": "70506", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 47." - }, - { - "id": "70507", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 48." - }, - { - "id": "70508", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 49." - }, - { - "id": "70509", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 50." - }, - { - "id": "70510", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 51." - }, - { - "id": "70511", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 52." - }, - { - "id": "70512", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 53." - }, - { - "id": "70513", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 54." - }, - { - "id": "70514", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 55." - }, - { - "id": "70515", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 56." - }, - { - "id": "70516", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 57." - }, - { - "id": "70517", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 58." - }, - { - "id": "70518", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 59." - }, - { - "id": "70519", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 60." - }, - { - "id": "70520", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 61." - }, - { - "id": "70521", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 62." - }, - { - "id": "70522", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 63." - }, - { - "id": "70523", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 64." - }, - { - "id": "70524", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 65." - }, - { - "id": "70525", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 66." - }, - { - "id": "70526", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 67." - }, - { - "id": "70527", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 68." - }, - { - "id": "70528", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 69." - }, - { - "id": "70529", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 70." - }, - { - "id": "70530", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 71." - }, - { - "id": "70531", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 72." - }, - { - "id": "70532", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 73." - }, - { - "id": "70533", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 74." - }, - { - "id": "70534", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 75." - }, - { - "id": "70535", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 76." - }, - { - "id": "70536", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 77." - }, - { - "id": "70537", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 78." - }, - { - "id": "70538", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 79." - }, - { - "id": "70539", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 80." - }, - { - "id": "70540", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 81." - }, - { - "id": "70541", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 82." - }, - { - "id": "70542", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 83." - }, - { - "id": "70543", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 84." - }, - { - "id": "70544", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 85." - }, - { - "id": "70545", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 86." - }, - { - "id": "70546", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 87." - }, - { - "id": "70547", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 88." - }, - { - "id": "70548", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 89." - }, - { - "id": "70549", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 90." - }, - { - "id": "70550", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 91." - }, - { - "id": "70551", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 92." - }, - { - "id": "70552", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 93." - }, - { - "id": "70553", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 94." - }, - { - "id": "70554", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 95." - }, - { - "id": "70555", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 96." - }, - { - "id": "70556", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 97." - }, - { - "id": "70557", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 98." - }, - { - "id": "70558", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 99." - }, - { - "id": "70559", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 100." - }, - { - "id": "70560", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 101." - }, - { - "id": "70561", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 102." - }, - { - "id": "70562", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 103." - }, - { - "id": "70563", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 104." - }, - { - "id": "70564", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 105." - }, - { - "id": "70565", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 106." - }, - { - "id": "70566", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 107." - }, - { - "id": "70567", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 108." - }, - { - "id": "70568", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 109." - }, - { - "id": "70569", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 110." - }, - { - "id": "70570", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 111." - }, - { - "id": "70571", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 112." - }, - { - "id": "70572", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 113." - }, - { - "id": "70573", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 114." - }, - { - "id": "70574", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 115." - }, - { - "id": "70575", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 116." - }, - { - "id": "70576", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 117." - }, - { - "id": "70577", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 118." - }, - { - "id": "70578", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 119." - }, - { - "id": "70579", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 120." - }, - { - "id": "70580", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 121." - }, - { - "id": "70581", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 122." - }, - { - "id": "70582", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 123." - }, - { - "id": "70583", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 124." - }, - { - "id": "70584", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 125." - }, - { - "id": "70585", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 126." - }, - { - "id": "70586", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 127." - }, - { - "id": "70587", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 128." - }, - { - "id": "70588", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 129." - }, - { - "id": "70589", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 130." - }, - { - "id": "70590", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 131." - }, - { - "id": "70591", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 132." - }, - { - "id": "70592", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 133." - }, - { - "id": "70593", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 134." - }, - { - "id": "70594", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 135." - }, - { - "id": "70595", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 136." - }, - { - "id": "70596", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 137." - }, - { - "id": "70597", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 138." - }, - { - "id": "70598", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 139." - }, - { - "id": "70599", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 140." - }, - { - "id": "70600", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 141." - }, - { - "id": "70601", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 142." - }, - { - "id": "70602", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 143." - }, - { - "id": "70603", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 144." - }, - { - "id": "70604", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: put this item on the agenda. No agreement was reached. Item 145." - }, - { - "id": "70605", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: rejected suggestion copied from a draft, awaiting a decision. Item 146." - }, - { - "id": "70606", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova access expiry: empty form awaiting the actual policy. Item 147." - }, - { - "id": "70607", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "Checking whether the team discussed nova access expiry. This is a question, not a confirmed decision. Item 148." - }, - { - "id": "70608", - "chatId": "204", - "senderId": "801", - "timestamp": "2026-10-06T12:00:00.000Z", - "text": "nova keyword list: nova access expiry; no instruction is recorded. Item 149." - } - ], - "queries": [ - { - "id": "aurora-question-decision", - "split": "dev", - "family": "aurora", - "category": "decision", - "retrieval": "aurora rollout rollback", - "text": "What rollback decision was agreed for the Aurora rollout?", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-question-question", - "split": "dev", - "family": "aurora", - "category": "question", - "retrieval": "aurora rollout rollback", - "text": "What should we do to roll back the Aurora rollout?", - "relevant": { - "10001": 2, - "10002": 1 - } - }, - { - "id": "aurora-question-no-answer", - "split": "dev", - "family": "aurora", - "category": "no-answer", - "retrieval": "aurora rollout rollback", - "text": "Which independent auditor signed off on aurora rollout rollback?", - "relevant": {} - }, - { - "id": "cedar-question-decision", - "split": "dev", - "family": "cedar", - "category": "decision", - "retrieval": "cedar backup retention", - "text": "What retention policy was agreed for Cedar backups?", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-question-question", - "split": "dev", - "family": "cedar", - "category": "question", - "retrieval": "cedar backup retention", - "text": "How long should Cedar backup snapshots be kept?", - "relevant": { - "10303": 2, - "10304": 1 - } - }, - { - "id": "cedar-question-no-answer", - "split": "dev", - "family": "cedar", - "category": "no-answer", - "retrieval": "cedar backup retention", - "text": "Which independent auditor signed off on cedar backup retention?", - "relevant": {} - }, - { - "id": "север-question-decision", - "split": "dev", - "family": "север", - "category": "decision", - "retrieval": "север сертификат обновление", - "text": "Какое решение принято по обновлению сертификата Север?", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-question-question", - "split": "dev", - "family": "север", - "category": "question", - "retrieval": "север сертификат обновление", - "text": "Что известно о новом сертификате Север и сроке его действия?", - "relevant": { - "10605": 2, - "10606": 1 - } - }, - { - "id": "север-question-no-answer", - "split": "dev", - "family": "север", - "category": "no-answer", - "retrieval": "север сертификат обновление", - "text": "Какой независимый аудитор подписал обновление сертификата Север?", - "relevant": {} - }, - { - "id": "lumen-question-decision", - "split": "dev", - "family": "lumen", - "category": "decision", - "retrieval": "lumen invoice refund", - "text": "What refund was approved for the Lumen invoice?", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-question-question", - "split": "dev", - "family": "lumen", - "category": "question", - "retrieval": "lumen invoice refund", - "text": "What should be returned to the payer of the Lumen invoice?", - "relevant": { - "10907": 2, - "10908": 1 - } - }, - { - "id": "lumen-question-no-answer", - "split": "dev", - "family": "lumen", - "category": "no-answer", - "retrieval": "lumen invoice refund", - "text": "Which independent auditor signed off on lumen invoice refund?", - "relevant": {} - }, - { - "id": "cobalt-question-decision", - "split": "test", - "family": "cobalt", - "category": "decision", - "retrieval": "cobalt failover routing", - "text": "What failover routing was approved for Cobalt?", - "relevant": { - "70001": 2, - "70002": 1 - } - }, - { - "id": "cobalt-question-question", - "split": "test", - "family": "cobalt", - "category": "question", - "retrieval": "cobalt failover routing", - "text": "Where should Cobalt traffic go after failover?", - "relevant": { - "70001": 2, - "70002": 1 - } - }, - { - "id": "cobalt-question-no-answer", - "split": "test", - "family": "cobalt", - "category": "no-answer", - "retrieval": "cobalt failover routing", - "text": "Which independent auditor signed off on cobalt failover routing?", - "relevant": {} - }, - { - "id": "birch-question-decision", - "split": "test", - "family": "birch", - "category": "decision", - "retrieval": "birch export encryption", - "text": "What export encryption policy was agreed for Birch?", - "relevant": { - "70153": 2, - "70154": 1 - } - }, - { - "id": "birch-question-question", - "split": "test", - "family": "birch", - "category": "question", - "retrieval": "birch export encryption", - "text": "How should Birch exports be encrypted before they are sent?", - "relevant": { - "70153": 2, - "70154": 1 - } - }, - { - "id": "birch-question-no-answer", - "split": "test", - "family": "birch", - "category": "no-answer", - "retrieval": "birch export encryption", - "text": "Which independent auditor signed off on birch export encryption?", - "relevant": {} - }, - { - "id": "спектр-question-decision", - "split": "test", - "family": "спектр", - "category": "decision", - "retrieval": "спектр очередь лимит", - "text": "Какой лимит очереди согласован для Спектр?", - "relevant": { - "70305": 2, - "70306": 1 - } - }, - { - "id": "спектр-question-question", - "split": "test", - "family": "спектр", - "category": "question", - "retrieval": "спектр очередь лимит", - "text": "Сколько задач принимает очередь Спектр и куда отправлять остальные?", - "relevant": { - "70305": 2, - "70306": 1 - } - }, - { - "id": "спектр-question-no-answer", - "split": "test", - "family": "спектр", - "category": "no-answer", - "retrieval": "спектр очередь лимит", - "text": "Какой независимый аудитор подписал лимит очереди Спектр?", - "relevant": {} - }, - { - "id": "nova-question-decision", - "split": "test", - "family": "nova", - "category": "decision", - "retrieval": "nova access expiry", - "text": "What access expiry policy was agreed for Nova?", - "relevant": { - "70457": 2, - "70458": 1 - } - }, - { - "id": "nova-question-question", - "split": "test", - "family": "nova", - "category": "question", - "retrieval": "nova access expiry", - "text": "When does Nova temporary access expire and how can it be extended?", - "relevant": { - "70457": 2, - "70458": 1 - } - }, - { - "id": "nova-question-no-answer", - "split": "test", - "family": "nova", - "category": "no-answer", - "retrieval": "nova access expiry", - "text": "Which independent auditor signed off on nova access expiry?", - "relevant": {} - } - ] -} diff --git a/bench/message-search-quality/reranker.json b/bench/message-search-quality/reranker.json deleted file mode 100644 index 7e6f1908d..000000000 --- a/bench/message-search-quality/reranker.json +++ /dev/null @@ -1,28 +0,0 @@ -{ - "repository": "cross-encoder/mmarco-mMiniLMv2-L12-H384-v1", - "revision": "1427fd652930e4ba29e8149678df786c240d8825", - "licence": "Apache-2.0", - "maxTokens": 512, - "files": [ - { - "name": "onnx/model_quint8_avx2.onnx", - "sha256": "6c2513767fb63d008a4377bef7a7a3555433d9436342bb53e35a3a72ffc52d4b", - "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/onnx/model_quint8_avx2.onnx" - }, - { - "name": "tokenizer.json", - "sha256": "62c24cdc13d4c9952d63718d6c9fa4c287974249e16b7ade6d5a85e7bbb75626", - "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/tokenizer.json" - }, - { - "name": "tokenizer_config.json", - "sha256": "e7fbfbfa6347b4e414c1cee50d142e2c2f9a895dad68b068ae83a8b564c3837e", - "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/tokenizer_config.json" - }, - { - "name": "config.json", - "sha256": "cc2cfe51aa3fd759d21d21acf5dfd6994aa67a3c9210636d22e143699d336c77", - "url": "https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1/resolve/1427fd652930e4ba29e8149678df786c240d8825/config.json" - } - ] -} diff --git a/bench/message-search-quality/resources.json b/bench/message-search-quality/resources.json deleted file mode 100644 index c17de332c..000000000 --- a/bench/message-search-quality/resources.json +++ /dev/null @@ -1 +0,0 @@ -{"model":{"id":"cross-encoder/mmarco-mMiniLMv2-L12-H384-v1","revision":"1427fd652930e4ba29e8149678df786c240d8825","licence":"Apache-2.0","files":[{"name":"onnx/model_quint8_avx2.onnx","sha256":"6c2513767fb63d008a4377bef7a7a3555433d9436342bb53e35a3a72ffc52d4b"},{"name":"tokenizer.json","sha256":"62c24cdc13d4c9952d63718d6c9fa4c287974249e16b7ade6d5a85e7bbb75626"},{"name":"tokenizer_config.json","sha256":"e7fbfbfa6347b4e414c1cee50d142e2c2f9a895dad68b068ae83a8b564c3837e"},{"name":"config.json","sha256":"cc2cfe51aa3fd759d21d21acf5dfd6994aa67a3c9210636d22e143699d336c77"}],"threads":8},"node":"v24.19.0","before":{"rss":70090752,"heapTotal":11898880,"heapUsed":10242648,"external":11704910,"arrayBuffers":143651},"afterLoad":{"rss":1055674368,"heapTotal":317984768,"heapUsed":178696576,"external":156148641,"arrayBuffers":147309097},"afterFirst":{"rss":1076772864,"heapTotal":318246912,"heapUsed":179462008,"external":156149173,"arrayBuffers":147309629},"after300":{"rss":1189621760,"heapTotal":318246912,"heapUsed":199818576,"external":127710196,"arrayBuffers":118870652},"peakObservedRss":1317421056,"processHighWaterBytes":1331576832,"modelIncrementAtLoad":985583616,"modelIncrementAfter300":1119531008,"loadMs":1270.7682869999999,"scoring300Ms":3247.012192,"score":8.382225036621094,"notes":"Entire isolated Node process RSS includes runtime, tokenizer, model weights and buffers; no store or account opened."} diff --git a/bench/message-search-quality/resources.ts b/bench/message-search-quality/resources.ts deleted file mode 100644 index ed1ad6c11..000000000 --- a/bench/message-search-quality/resources.ts +++ /dev/null @@ -1,48 +0,0 @@ -import { readFileSync, existsSync } from "node:fs" -import { performance } from "node:perf_hooks" -import { openCrossEncoder } from "./crossencoder.ts" -const before = process.memoryUsage() -const started = performance.now() -const directory = process.env.MESSAGE_RERANKER_DIR -if (!directory) throw new Error("MESSAGE_RERANKER_DIR must name the separately prepared local model") -const ranker = await openCrossEncoder(directory) -const afterLoad = process.memoryUsage() -const loadMs = performance.now() - started -const modelScore = await ranker.score( - "What rollback decision was agreed for the Aurora rollout?", - "Aurora rollout rollback decision: disable the canary and restore build 17.", -) -const afterFirst = process.memoryUsage() -let peakRss = afterFirst.rss -const start = performance.now() -for (let i = 0; i < 300; i++) { - await ranker.score( - "What rollback decision was agreed for the Aurora rollout?", - `Aurora rollout rollback: agenda only; no decision yet. Thread ${i}.`, - ) - peakRss = Math.max(peakRss, process.memoryUsage().rss) -} -const after300 = process.memoryUsage() -const scoring300Ms = performance.now() - start -const highWaterKiB = existsSync("/proc/self/status") - ? Number(/^VmHWM:\s+(\d+)/m.exec(readFileSync("/proc/self/status", "utf8"))?.[1]) - : process.resourceUsage().maxRSS -await ranker.close() -const report = { - model: ranker.model, - node: process.version, - before, - afterLoad, - afterFirst, - after300, - peakObservedRss: peakRss, - processHighWaterBytes: highWaterKiB * 1024, - modelIncrementAtLoad: afterLoad.rss - before.rss, - modelIncrementAfter300: after300.rss - before.rss, - loadMs, - scoring300Ms, - score: modelScore, - notes: - "Entire isolated Node process RSS includes runtime, tokenizer, model weights and buffers; no store or account opened.", -} -console.log(JSON.stringify(report)) diff --git a/bench/message-search-quality/run.ts b/bench/message-search-quality/run.ts deleted file mode 100644 index 6f8e83bb8..000000000 --- a/bench/message-search-quality/run.ts +++ /dev/null @@ -1,137 +0,0 @@ -import assert from "node:assert/strict" -import { createHash } from "node:crypto" -import { mkdtempSync, readFileSync } from "node:fs" -import { tmpdir } from "node:os" -import { join } from "node:path" -import { fileURLToPath } from "node:url" -import type { Message } from "../../dist/domain/models.js" -import { searchStore } from "../../dist/services/messages.js" -import { openStore } from "../../dist/store/store.js" - -type Mode = "lucene" | "legacy" -type Query = { - id: string - split: "dev" | "test" - category: string - intent: string - lucene: string - legacy: string - relevant: Record -} -type Corpus = { - chats: { id: string; title: string }[] - messages: Pick[] - queries: Query[] -} -type Row = { - id: string - split: Query["split"] - category: string - mode: Mode - text: string - relevant: Record - hits: { id: string; chatId: string; match: string | null; score: number | null }[] - corrections: { from: string; to: string[] }[] - recallAt10: number | null - mrr: number | null - ndcgAt10: number | null - noAnswerFalseHits: number -} - -const corpusBytes = readFileSync(new URL("./corpus.json", import.meta.url)) -const corpus = JSON.parse(corpusBytes.toString()) as Corpus -const supplementBytes = process.argv.includes("--include-questions") ? readFileSync(new URL("./questions.json", import.meta.url)) : undefined -if (supplementBytes) { - const supplement = JSON.parse(supplementBytes.toString()) as Pick - corpus.chats.push(...supplement.chats) - corpus.messages.push(...supplement.messages) -} -const ids = new Set(corpus.messages.map(({ id }) => id)) -assert.equal(ids.size, corpus.messages.length) -assert.equal(new Set(corpus.queries.map(({ id }) => id)).size, corpus.queries.length) -for (const q of corpus.queries) { - for (const [id, grade] of Object.entries(q.relevant)) { - assert.ok(ids.has(id), `unknown relevance label ${id}`) - assert.ok(grade === 1 || grade === 2) - } -} -const root = mkdtempSync(join(tmpdir(), "message-search-quality-")) -const path = join(root, "quality.db") -// Set this before opening any store; the explicit path remains the authority. -process.env.TZ = "UTC" -process.env.MESSAGING_STORE = path -process.env.CLI_COMMON_CACHE_DIR = join(root, "empty-cache") -const account = { provider: "synthetic", account: "500" } -const store = await openStore({ path }) -const rows: Row[] = [] -try { - await store.saveChats(account, corpus.chats.map((chat) => ({ - ...chat, kind: "group", unreadCount: 0, lastMessageAt: "2026-10-04T12:00:00.000Z", participantsCount: 100, - }))) - for (const chat of corpus.chats) { - await store.saveMessages(account, chat.id, corpus.messages.filter((m) => m.chatId === chat.id).map((m): Message => ({ - ...m, senderName: `Synthetic ${m.senderId}`, editedAt: null, outgoing: false, attachments: [], - replyTo: null, forwardedFrom: null, reactions: null, - })), { via: "synthetic" }) - } - await store.fillSearchIndex({}) - await store.fillStems({}) - assert.equal((await store.searchIndexState())?.ready, true) - for (const q of corpus.queries) { - for (const mode of ["lucene", "legacy"] as const) { - const found = await searchStore(store, account, { - text: q[mode], language: mode, limit: 10, - ...(mode === "lucene" ? { timezone: "UTC" } : {}), - }) - assert.ok(found.wordsReady) - if (mode === "lucene") assert.ok(found.stemsReady) - const hits = found.items.map((h) => ({ id: h.id, chatId: h.chatId, match: h.match ?? null, score: h.score ?? null })) - assert.equal(new Set(hits.map((h) => h.id)).size, hits.length) - assert.ok(hits.every((h) => ids.has(h.id))) - const grades = Object.values(q.relevant) - const relevant = hits.filter((h) => q.relevant[h.id]).length - const first = hits.findIndex((h) => q.relevant[h.id]) - const dcg = (g: number[]) => g.reduce((sum, grade, i) => sum + (2 ** grade - 1) / Math.log2(i + 2), 0) - const ideal = dcg([...grades].sort((a, b) => b - a).slice(0, 10)) - rows.push({ - id: q.id, split: q.split, category: q.category, mode, text: q[mode], relevant: q.relevant, hits, - corrections: found.corrections, - recallAt10: grades.length ? relevant / grades.length : null, - mrr: grades.length ? first < 0 ? 0 : 1 / (first + 1) : null, - ndcgAt10: grades.length ? dcg(hits.map((h) => q.relevant[h.id] ?? 0)) / ideal : null, - noAnswerFalseHits: grades.length ? 0 : hits.length, - }) - } - } - assert.equal(rows.length, corpus.queries.length * 2) - const summarize = (selected: Row[]) => { - const answerable = selected.filter((r) => r.recallAt10 !== null) - const negatives = selected.filter((r) => r.recallAt10 === null) - const mean = (key: "recallAt10" | "mrr" | "ndcgAt10") => answerable.length - ? answerable.reduce((sum, r) => sum + (r[key] ?? 0), 0) / answerable.length : null - return { - queries: selected.length, answerable: answerable.length, noAnswerQueries: negatives.length, - recallAt10: mean("recallAt10"), mrr: mean("mrr"), ndcgAt10: mean("ndcgAt10"), - noAnswerFalseHits: negatives.reduce((sum, r) => sum + r.noAnswerFalseHits, 0), - noAnswerQueryHitRate: negatives.length ? negatives.filter((r) => r.hits.length > 0).length / negatives.length : null, - } - } - const digest = (bytes: Buffer) => createHash("sha256").update(bytes).digest("hex") - const productionFiles = ["services/messages.js", "services/messages-search.js", "search/search.js", "search/query.js", - "search/correct.js", "search/stem.js", "search/lucene/parser.js", "store/sqlite/lucene.js", "store/sqlite/words.js"] - console.log(JSON.stringify({ - schemaVersion: 1, k: 10, node: process.version, timezone: "UTC", storePath: path, - archiveSupplementSha256: supplementBytes ? digest(supplementBytes) : null, - corpusSha256: digest(corpusBytes), runnerSha256: digest(readFileSync(fileURLToPath(import.meta.url))), - productionSha256: Object.fromEntries(productionFiles.map((file) => [file, digest(readFileSync(new URL(`../../dist/${file}`, import.meta.url)))])), - messages: corpus.messages.length, queries: corpus.queries.length, - metrics: Object.fromEntries(["dev", "test"].map((split) => [split, Object.fromEntries(["lucene", "legacy"].map((mode) => { - const selected = rows.filter((r) => r.split === split && r.mode === mode) - return [mode, { ...summarize(selected), categories: Object.fromEntries([...new Set(selected.map((r) => r.category))] - .map((category) => [category, summarize(selected.filter((r) => r.category === category))])) }] - }))])), - rankings: rows, - }, null, 2)) -} finally { - await store.close() -} diff --git a/bench/message-search-quality/semantic-dev.json b/bench/message-search-quality/semantic-dev.json deleted file mode 100644 index 2adec9716..000000000 --- a/bench/message-search-quality/semantic-dev.json +++ /dev/null @@ -1,27423 +0,0 @@ -{ - "crossEncoderImplementationSha256": "2d044b35dc1f4ac769127e1b86303b5b4d20f7970eb2ea9a47df729153e96aa0", - "rerankerManifestSha256": "8efb505319d19cb42772dc02438cd99c68cfd4771bcba8890aac3ec81b1538d3", - "corpusSha256": "31ffb8689e06d315a9be7d2ba0094a859b05cb67fc17eb68e4b5b4f5744a0b20", - "questionsSha256": "390ff1475964d982a06afaf9e2f6f840f5b8304fcffe58ebedd5a0ea8d54478e", - "runnerSha256": "bea007961d0dc1d237824c274029f81dce28c4840e8df88c9b2b282d9bf07fd5", - "productionSha256": { - "services/messages-combined.js": "d10aeb5f7049f615503d59c37df109bcd284f53b5cfceaac0151f2d7e2ca60e9", - "search/lucene/parser.js": "5e5562c7512785a06c15bfa05915618c24e0eb7dfac9f73d502958cf02705184", - "embeddings/embed.js": "cacf0bec8b98aeda4ed135642d593c4d5b50800867bc8e95de0ad77c1e002e3f", - "embeddings/models.js": "8debfaa01509728ba206702f27e6d8251b8968a03375089b6ca1af9d0eb53e94", - "store/sqlite/lucene.js": "efd3f9909ac8205b02019160be2234821f4dacba26d01dc754c1e7e888173964" - }, - "node": "v24.19.0", - "cpu": "AMD Ryzen AI 9 HX 470 w/ Radeon 890M", - "storePath": "/tmp/semantic-message-quality-2H4Bdn/quality.db", - "model": { - "id": "e5-small", - "files": [ - { - "name": "onnx/model_quantized.onnx", - "sha256": "f80102d3f2a1229f387d3c81909990d8945513e347b0eab049f7de3c6f98c193" - }, - { - "name": "tokenizer.json", - "sha256": "0b44a9d7b51c3c62626640cda0e2c2f70fdacdc25bbbd68038369d14ebdf4c39" - }, - { - "name": "tokenizer_config.json", - "sha256": "a1d6bc8734a6f635dc158508bef000f8e2e5a759c7d92f984b2c86e5ff53425b" - } - ], - "threads": 8 - }, - "kind": "embedding-cosine reranking; not a cross-encoder", - "candidateSummary": [ - { - "depth": 50, - "candidateRecall": 0.4375, - "byCategory": { - "words": 0.25, - "typo": 0.25, - "beginnings": 0.25, - "phrase": 0.125, - "from": 1, - "chat": 1, - "date": 1, - "and": 0.25, - "or": 0.25, - "paraphrase": 0 - } - }, - { - "depth": 100, - "candidateRecall": 0.75, - "byCategory": { - "words": 0.75, - "typo": 0.75, - "beginnings": 0.75, - "phrase": 0.75, - "from": 1, - "chat": 1, - "date": 1, - "and": 0.75, - "or": 0.75, - "paraphrase": 0 - } - }, - { - "depth": 300, - "candidateRecall": 0.9, - "byCategory": { - "words": 1, - "typo": 1, - "beginnings": 1, - "phrase": 1, - "from": 1, - "chat": 1, - "date": 1, - "and": 1, - "or": 1, - "paraphrase": 0 - } - }, - { - "depth": 500, - "candidateRecall": 0.9, - "byCategory": { - "words": 1, - "typo": 1, - "beginnings": 1, - "phrase": 1, - "from": 1, - "chat": 1, - "date": 1, - "and": 1, - "or": 1, - "paraphrase": 0 - } - } - ], - "selectedOnDev": { - "depth": 300, - "questionScoreFloor": 0.9 - }, - "keywordSummary": { - "lexical": { - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.3625, - "mrr": 0.375, - "ndcgAt10": 0.3263914259579807, - "noAnswerFalseHits": 0 - }, - "semantic": { - "answerable": 40, - "noAnswerQueries": 8, - "recallAt10": 0.275, - "mrr": 0.3, - "ndcgAt10": 0.26737653428714403, - "noAnswerFalseHits": 0 - } - }, - "questionSummary": { - "direct": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - }, - "anchoredLexical": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.25, - "mrr": 0.25, - "ndcgAt10": 0.19917689524762666, - "noAnswerFalseHits": 40 - }, - "sweep": [ - { - "floor": null, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.625, - "ndcgAt10": 0.5963853267110182, - "noAnswerFalseHits": 40 - } - }, - { - "floor": 0.7, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.625, - "ndcgAt10": 0.5963853267110182, - "noAnswerFalseHits": 40 - } - }, - { - "floor": 0.75, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.625, - "ndcgAt10": 0.5963853267110182, - "noAnswerFalseHits": 40 - } - }, - { - "floor": 0.8, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.625, - "ndcgAt10": 0.5963853267110182, - "noAnswerFalseHits": 40 - } - }, - { - "floor": 0.85, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0.5625, - "mrr": 0.625, - "ndcgAt10": 0.5963853267110182, - "noAnswerFalseHits": 35 - } - }, - { - "floor": 0.9, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - } - }, - { - "floor": 0.95, - "metrics": { - "answerable": 8, - "noAnswerQueries": 4, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "noAnswerFalseHits": 0 - } - } - ] - }, - "devQualified": false, - "questionRetrieval": "manual topic anchors for diagnostic reranking; full-question direct retrieval is measured separately", - "heldOut": null, - "inference": { - "pairInferenceMs": 0, - "scoredPairs": 0, - "averagePairMs": null, - "loadMs": 1107.57413, - "embeddedMessages": 1200, - "uniqueEmbeddedQueries": 48, - "uniqueCrossEncoderQueries": 0, - "passageInferenceMs": 12019.547399, - "queryInferenceMs": 356.87080400000104, - "amortizedPerMessageMs": 10.016289499166666, - "cache": "new in-memory synthetic message vectors; no persistent production message index" - }, - "candidateRows": [ - { - "id": "aurora-words", - "category": "words", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-words", - "category": "words", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-words", - "category": "words", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-words", - "category": "words", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 50, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 100, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 300, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-from", - "category": "from", - "depth": 500, - "relevant": { - "10001": 2 - }, - "candidateIds": [ - "10001" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-date", - "category": "date", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10001", - "10002" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-and", - "category": "and", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-and", - "category": "and", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-and", - "category": "and", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-and", - "category": "and", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-or", - "category": "or", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-or", - "category": "or", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-or", - "category": "or", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "aurora-or", - "category": "or", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [ - "10298", - "10293", - "10288", - "10283", - "10278", - "10273", - "10268", - "10263", - "10258", - "10253", - "10248", - "10243", - "10238", - "10233", - "10228", - "10223", - "10218", - "10213", - "10208", - "10203", - "10198", - "10193", - "10188", - "10183", - "10178", - "10173", - "10168", - "10163", - "10158", - "10153", - "10148", - "10143", - "10138", - "10133", - "10128", - "10123", - "10118", - "10113", - "10108", - "10103", - "10098", - "10093", - "10088", - "10083", - "10078", - "10073", - "10068", - "10063", - "10058", - "10053", - "10048", - "10043", - "10038", - "10033", - "10028", - "10023", - "10018", - "10013", - "10008", - "10003", - "10001", - "10299", - "10294", - "10289", - "10284", - "10279", - "10274", - "10269", - "10264", - "10259", - "10254", - "10249", - "10244", - "10239", - "10234", - "10229", - "10224", - "10219", - "10214", - "10209", - "10204", - "10199", - "10194", - "10189", - "10184", - "10179", - "10174", - "10169", - "10164", - "10159", - "10154", - "10149", - "10144", - "10139", - "10134", - "10129", - "10124", - "10119", - "10114", - "10109", - "10104", - "10099", - "10094", - "10089", - "10084", - "10079", - "10074", - "10069", - "10064", - "10059", - "10054", - "10049", - "10044", - "10039", - "10034", - "10029", - "10024", - "10019", - "10014", - "10009", - "10004", - "10002", - "10300", - "10295", - "10290", - "10285", - "10280", - "10275", - "10270", - "10265", - "10260", - "10255", - "10250", - "10245", - "10240", - "10235", - "10230", - "10225", - "10220", - "10215", - "10210", - "10205", - "10200", - "10195", - "10190", - "10185", - "10180", - "10175", - "10170", - "10165", - "10160", - "10155", - "10150", - "10145", - "10140", - "10135", - "10130", - "10125", - "10120", - "10115", - "10110", - "10105", - "10100", - "10095", - "10090", - "10085", - "10080", - "10075", - "10070", - "10065", - "10060", - "10055", - "10050", - "10045", - "10040", - "10035", - "10030", - "10025", - "10020", - "10015", - "10010", - "10005", - "10302", - "10297", - "10292", - "10287", - "10282", - "10277", - "10272", - "10267", - "10262", - "10257", - "10252", - "10247", - "10242", - "10237", - "10232", - "10227", - "10222", - "10217", - "10212", - "10207", - "10202", - "10197", - "10192", - "10187", - "10182", - "10177", - "10172", - "10167", - "10162", - "10157", - "10152", - "10147", - "10142", - "10137", - "10132", - "10127", - "10122", - "10117", - "10112", - "10107", - "10102", - "10097", - "10092", - "10087", - "10082", - "10077", - "10072", - "10067", - "10062", - "10057", - "10052", - "10047", - "10042", - "10037", - "10032", - "10027", - "10022", - "10017", - "10012", - "10007", - "10301", - "10296", - "10291", - "10286", - "10281", - "10276", - "10271", - "10266", - "10261", - "10256", - "10251", - "10246", - "10241", - "10236", - "10231", - "10226", - "10221", - "10216", - "10211", - "10206", - "10201", - "10196", - "10191", - "10186", - "10181", - "10176", - "10171", - "10166", - "10161", - "10156", - "10151", - "10146", - "10141", - "10136", - "10131", - "10126", - "10121", - "10116", - "10111", - "10106", - "10101", - "10096", - "10091", - "10086", - "10081", - "10076", - "10071", - "10066", - "10061", - "10056", - "10051", - "10046", - "10041", - "10036", - "10031", - "10026", - "10021", - "10016", - "10011", - "10006" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10001": 2, - "10002": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-words", - "category": "words", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-words", - "category": "words", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-words", - "category": "words", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-words", - "category": "words", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 50, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 100, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 300, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-from", - "category": "from", - "depth": 500, - "relevant": { - "10303": 2 - }, - "candidateIds": [ - "10303" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-date", - "category": "date", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10303", - "10304" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-and", - "category": "and", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-and", - "category": "and", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-and", - "category": "and", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-and", - "category": "and", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-or", - "category": "or", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-or", - "category": "or", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411" - ], - "candidateRecall": 0.5, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-or", - "category": "or", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "cedar-or", - "category": "or", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [ - "10600", - "10595", - "10590", - "10585", - "10580", - "10575", - "10570", - "10565", - "10560", - "10555", - "10550", - "10545", - "10540", - "10535", - "10530", - "10525", - "10520", - "10515", - "10510", - "10505", - "10500", - "10495", - "10490", - "10485", - "10480", - "10475", - "10470", - "10465", - "10460", - "10455", - "10450", - "10445", - "10440", - "10435", - "10430", - "10425", - "10420", - "10415", - "10410", - "10405", - "10400", - "10395", - "10390", - "10385", - "10380", - "10375", - "10370", - "10365", - "10360", - "10355", - "10350", - "10345", - "10340", - "10335", - "10330", - "10325", - "10320", - "10315", - "10310", - "10305", - "10303", - "10601", - "10596", - "10591", - "10586", - "10581", - "10576", - "10571", - "10566", - "10561", - "10556", - "10551", - "10546", - "10541", - "10536", - "10531", - "10526", - "10521", - "10516", - "10511", - "10506", - "10501", - "10496", - "10491", - "10486", - "10481", - "10476", - "10471", - "10466", - "10461", - "10456", - "10451", - "10446", - "10441", - "10436", - "10431", - "10426", - "10421", - "10416", - "10411", - "10406", - "10401", - "10396", - "10391", - "10386", - "10381", - "10376", - "10371", - "10366", - "10361", - "10356", - "10351", - "10346", - "10341", - "10336", - "10331", - "10326", - "10321", - "10316", - "10311", - "10306", - "10304", - "10602", - "10597", - "10592", - "10587", - "10582", - "10577", - "10572", - "10567", - "10562", - "10557", - "10552", - "10547", - "10542", - "10537", - "10532", - "10527", - "10522", - "10517", - "10512", - "10507", - "10502", - "10497", - "10492", - "10487", - "10482", - "10477", - "10472", - "10467", - "10462", - "10457", - "10452", - "10447", - "10442", - "10437", - "10432", - "10427", - "10422", - "10417", - "10412", - "10407", - "10402", - "10397", - "10392", - "10387", - "10382", - "10377", - "10372", - "10367", - "10362", - "10357", - "10352", - "10347", - "10342", - "10337", - "10332", - "10327", - "10322", - "10317", - "10312", - "10307", - "10604", - "10599", - "10594", - "10589", - "10584", - "10579", - "10574", - "10569", - "10564", - "10559", - "10554", - "10549", - "10544", - "10539", - "10534", - "10529", - "10524", - "10519", - "10514", - "10509", - "10504", - "10499", - "10494", - "10489", - "10484", - "10479", - "10474", - "10469", - "10464", - "10459", - "10454", - "10449", - "10444", - "10439", - "10434", - "10429", - "10424", - "10419", - "10414", - "10409", - "10404", - "10399", - "10394", - "10389", - "10384", - "10379", - "10374", - "10369", - "10364", - "10359", - "10354", - "10349", - "10344", - "10339", - "10334", - "10329", - "10324", - "10319", - "10314", - "10309", - "10603", - "10598", - "10593", - "10588", - "10583", - "10578", - "10573", - "10568", - "10563", - "10558", - "10553", - "10548", - "10543", - "10538", - "10533", - "10528", - "10523", - "10518", - "10513", - "10508", - "10503", - "10498", - "10493", - "10488", - "10483", - "10478", - "10473", - "10468", - "10463", - "10458", - "10453", - "10448", - "10443", - "10438", - "10433", - "10428", - "10423", - "10418", - "10413", - "10408", - "10403", - "10398", - "10393", - "10388", - "10383", - "10378", - "10373", - "10368", - "10363", - "10358", - "10353", - "10348", - "10343", - "10338", - "10333", - "10328", - "10323", - "10318", - "10313", - "10308" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10303": 2, - "10304": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-words", - "category": "words", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-words", - "category": "words", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-words", - "category": "words", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-words", - "category": "words", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662" - ], - "candidateRecall": 0.5, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10605", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10605", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611" - ], - "candidateRecall": 1, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10605", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611" - ], - "candidateRecall": 1, - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 50, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 100, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 300, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-from", - "category": "from", - "depth": 500, - "relevant": { - "10605": 2 - }, - "candidateIds": [ - "10605" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-date", - "category": "date", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10605", - "10606" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-and", - "category": "and", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-and", - "category": "and", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-and", - "category": "and", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-and", - "category": "and", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-or", - "category": "or", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-or", - "category": "or", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-or", - "category": "or", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": true - }, - { - "id": "север-or", - "category": "or", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [ - "10606", - "10605", - "10902", - "10897", - "10892", - "10887", - "10882", - "10877", - "10872", - "10867", - "10862", - "10857", - "10852", - "10847", - "10842", - "10837", - "10832", - "10827", - "10822", - "10817", - "10812", - "10807", - "10802", - "10797", - "10792", - "10787", - "10782", - "10777", - "10772", - "10767", - "10762", - "10757", - "10752", - "10747", - "10742", - "10737", - "10732", - "10727", - "10722", - "10717", - "10712", - "10707", - "10702", - "10697", - "10692", - "10687", - "10682", - "10677", - "10672", - "10667", - "10662", - "10657", - "10652", - "10647", - "10642", - "10637", - "10632", - "10627", - "10622", - "10617", - "10612", - "10607", - "10903", - "10898", - "10893", - "10888", - "10883", - "10878", - "10873", - "10868", - "10863", - "10858", - "10853", - "10848", - "10843", - "10838", - "10833", - "10828", - "10823", - "10818", - "10813", - "10808", - "10803", - "10798", - "10793", - "10788", - "10783", - "10778", - "10773", - "10768", - "10763", - "10758", - "10753", - "10748", - "10743", - "10738", - "10733", - "10728", - "10723", - "10718", - "10713", - "10708", - "10703", - "10698", - "10693", - "10688", - "10683", - "10678", - "10673", - "10668", - "10663", - "10658", - "10653", - "10648", - "10643", - "10638", - "10633", - "10628", - "10623", - "10618", - "10613", - "10608", - "10904", - "10899", - "10894", - "10889", - "10884", - "10879", - "10874", - "10869", - "10864", - "10859", - "10854", - "10849", - "10844", - "10839", - "10834", - "10829", - "10824", - "10819", - "10814", - "10809", - "10804", - "10799", - "10794", - "10789", - "10784", - "10779", - "10774", - "10769", - "10764", - "10759", - "10754", - "10749", - "10744", - "10739", - "10734", - "10729", - "10724", - "10719", - "10714", - "10709", - "10704", - "10699", - "10694", - "10689", - "10684", - "10679", - "10674", - "10669", - "10664", - "10659", - "10654", - "10649", - "10644", - "10639", - "10634", - "10629", - "10624", - "10619", - "10614", - "10609", - "10906", - "10901", - "10896", - "10891", - "10886", - "10881", - "10876", - "10871", - "10866", - "10861", - "10856", - "10851", - "10846", - "10841", - "10836", - "10831", - "10826", - "10821", - "10816", - "10811", - "10806", - "10801", - "10796", - "10791", - "10786", - "10781", - "10776", - "10771", - "10766", - "10761", - "10756", - "10751", - "10746", - "10741", - "10736", - "10731", - "10726", - "10721", - "10716", - "10711", - "10706", - "10701", - "10696", - "10691", - "10686", - "10681", - "10676", - "10671", - "10666", - "10661", - "10656", - "10651", - "10646", - "10641", - "10636", - "10631", - "10626", - "10621", - "10616", - "10611", - "10905", - "10900", - "10895", - "10890", - "10885", - "10880", - "10875", - "10870", - "10865", - "10860", - "10855", - "10850", - "10845", - "10840", - "10835", - "10830", - "10825", - "10820", - "10815", - "10810", - "10805", - "10800", - "10795", - "10790", - "10785", - "10780", - "10775", - "10770", - "10765", - "10760", - "10755", - "10750", - "10745", - "10740", - "10735", - "10730", - "10725", - "10720", - "10715", - "10710", - "10705", - "10700", - "10695", - "10690", - "10685", - "10680", - "10675", - "10670", - "10665", - "10660", - "10655", - "10650", - "10645", - "10640", - "10635", - "10630", - "10625", - "10620", - "10615", - "10610" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10605": 2, - "10606": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-words", - "category": "words", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-words", - "category": "words", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-words", - "category": "words", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-words", - "category": "words", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-typo", - "category": "typo", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-phrase", - "category": "phrase", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 50, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 100, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 300, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-from", - "category": "from", - "depth": 500, - "relevant": { - "10907": 2 - }, - "candidateIds": [ - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-chat", - "category": "chat", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-date", - "category": "date", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "10908", - "10907" - ], - "candidateRecall": 1, - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-and", - "category": "and", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-and", - "category": "and", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-and", - "category": "and", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-and", - "category": "and", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-or", - "category": "or", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959" - ], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-or", - "category": "or", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-or", - "category": "or", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": true - }, - { - "id": "lumen-or", - "category": "or", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [ - "11204", - "11199", - "11194", - "11189", - "11184", - "11179", - "11174", - "11169", - "11164", - "11159", - "11154", - "11149", - "11144", - "11139", - "11134", - "11129", - "11124", - "11119", - "11114", - "11109", - "11104", - "11099", - "11094", - "11089", - "11084", - "11079", - "11074", - "11069", - "11064", - "11059", - "11054", - "11049", - "11044", - "11039", - "11034", - "11029", - "11024", - "11019", - "11014", - "11009", - "11004", - "10999", - "10994", - "10989", - "10984", - "10979", - "10974", - "10969", - "10964", - "10959", - "10954", - "10949", - "10944", - "10939", - "10934", - "10929", - "10924", - "10919", - "10914", - "10909", - "10908", - "10907", - "11205", - "11200", - "11195", - "11190", - "11185", - "11180", - "11175", - "11170", - "11165", - "11160", - "11155", - "11150", - "11145", - "11140", - "11135", - "11130", - "11125", - "11120", - "11115", - "11110", - "11105", - "11100", - "11095", - "11090", - "11085", - "11080", - "11075", - "11070", - "11065", - "11060", - "11055", - "11050", - "11045", - "11040", - "11035", - "11030", - "11025", - "11020", - "11015", - "11010", - "11005", - "11000", - "10995", - "10990", - "10985", - "10980", - "10975", - "10970", - "10965", - "10960", - "10955", - "10950", - "10945", - "10940", - "10935", - "10930", - "10925", - "10920", - "10915", - "10910", - "11206", - "11201", - "11196", - "11191", - "11186", - "11181", - "11176", - "11171", - "11166", - "11161", - "11156", - "11151", - "11146", - "11141", - "11136", - "11131", - "11126", - "11121", - "11116", - "11111", - "11106", - "11101", - "11096", - "11091", - "11086", - "11081", - "11076", - "11071", - "11066", - "11061", - "11056", - "11051", - "11046", - "11041", - "11036", - "11031", - "11026", - "11021", - "11016", - "11011", - "11006", - "11001", - "10996", - "10991", - "10986", - "10981", - "10976", - "10971", - "10966", - "10961", - "10956", - "10951", - "10946", - "10941", - "10936", - "10931", - "10926", - "10921", - "10916", - "10911", - "11208", - "11203", - "11198", - "11193", - "11188", - "11183", - "11178", - "11173", - "11168", - "11163", - "11158", - "11153", - "11148", - "11143", - "11138", - "11133", - "11128", - "11123", - "11118", - "11113", - "11108", - "11103", - "11098", - "11093", - "11088", - "11083", - "11078", - "11073", - "11068", - "11063", - "11058", - "11053", - "11048", - "11043", - "11038", - "11033", - "11028", - "11023", - "11018", - "11013", - "11008", - "11003", - "10998", - "10993", - "10988", - "10983", - "10978", - "10973", - "10968", - "10963", - "10958", - "10953", - "10948", - "10943", - "10938", - "10933", - "10928", - "10923", - "10918", - "10913", - "11207", - "11202", - "11197", - "11192", - "11187", - "11182", - "11177", - "11172", - "11167", - "11162", - "11157", - "11152", - "11147", - "11142", - "11137", - "11132", - "11127", - "11122", - "11117", - "11112", - "11107", - "11102", - "11097", - "11092", - "11087", - "11082", - "11077", - "11072", - "11067", - "11062", - "11057", - "11052", - "11047", - "11042", - "11037", - "11032", - "11027", - "11022", - "11017", - "11012", - "11007", - "11002", - "10997", - "10992", - "10987", - "10982", - "10977", - "10972", - "10967", - "10962", - "10957", - "10952", - "10947", - "10942", - "10937", - "10932", - "10927", - "10922", - "10917", - "10912" - ], - "candidateRecall": 1, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 50, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 100, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 300, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "depth": 500, - "relevant": { - "10907": 2, - "10908": 1 - }, - "candidateIds": [], - "candidateRecall": 0, - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 50, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 100, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 300, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "depth": 500, - "relevant": {}, - "candidateIds": [], - "candidateRecall": null, - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0, - "truncated": false - } - ], - "keywordRows": [ - { - "id": "aurora-words", - "category": "words", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10280", - "score": 0.9192052184191694, - "lexicalRank": 126 - }, - { - "id": "10190", - "score": 0.919062760430714, - "lexicalRank": 144 - }, - { - "id": "10065", - "score": 0.9186915386380173, - "lexicalRank": 169 - }, - { - "id": "10225", - "score": 0.9174911203968827, - "lexicalRank": 137 - }, - { - "id": "10125", - "score": 0.9169708122949837, - "lexicalRank": 157 - }, - { - "id": "10075", - "score": 0.9168015790069396, - "lexicalRank": 167 - }, - { - "id": "10060", - "score": 0.9167733798319815, - "lexicalRank": 170 - }, - { - "id": "10230", - "score": 0.9164214060408934, - "lexicalRank": 136 - }, - { - "id": "10165", - "score": 0.9161771390615109, - "lexicalRank": 149 - }, - { - "id": "10270", - "score": 0.91574480912613, - "lexicalRank": 128 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-typo", - "category": "typo", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10156", - "score": 0.9041150488737639, - "lexicalRank": 271 - }, - { - "id": "10076", - "score": 0.904086260962625, - "lexicalRank": 287 - }, - { - "id": "10166", - "score": 0.9040358962609147, - "lexicalRank": 269 - }, - { - "id": "10065", - "score": 0.903079943144814, - "lexicalRank": 169 - }, - { - "id": "10280", - "score": 0.9028932023824128, - "lexicalRank": 126 - }, - { - "id": "10225", - "score": 0.9028142635467321, - "lexicalRank": 137 - }, - { - "id": "10151", - "score": 0.9024972208063297, - "lexicalRank": 272 - }, - { - "id": "10281", - "score": 0.9022675555094835, - "lexicalRank": 246 - }, - { - "id": "10190", - "score": 0.9021813279751438, - "lexicalRank": 144 - }, - { - "id": "10085", - "score": 0.9020156894563678, - "lexicalRank": 165 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-beginnings", - "category": "beginnings", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10280", - "score": 0.8978866948381442, - "lexicalRank": 126 - }, - { - "id": "10156", - "score": 0.8965793616743678, - "lexicalRank": 271 - }, - { - "id": "10076", - "score": 0.8964982852524149, - "lexicalRank": 287 - }, - { - "id": "10166", - "score": 0.8956208111669557, - "lexicalRank": 269 - }, - { - "id": "10190", - "score": 0.8954033568937041, - "lexicalRank": 144 - }, - { - "id": "10075", - "score": 0.8952151611051157, - "lexicalRank": 167 - }, - { - "id": "10225", - "score": 0.8952053102390007, - "lexicalRank": 137 - }, - { - "id": "10065", - "score": 0.8951843807049377, - "lexicalRank": 169 - }, - { - "id": "10301", - "score": 0.8948241205972687, - "lexicalRank": 242 - }, - { - "id": "10111", - "score": 0.8945984704883156, - "lexicalRank": 280 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-phrase", - "category": "phrase", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 2.7623348535297314 - }, - { - "id": "10293", - "score": 2.7623348535297314 - }, - { - "id": "10288", - "score": 2.7623348535297314 - }, - { - "id": "10283", - "score": 2.7623348535297314 - }, - { - "id": "10278", - "score": 2.7623348535297314 - }, - { - "id": "10273", - "score": 2.7623348535297314 - }, - { - "id": "10268", - "score": 2.7623348535297314 - }, - { - "id": "10263", - "score": 2.7623348535297314 - }, - { - "id": "10258", - "score": 2.7623348535297314 - }, - { - "id": "10253", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "10298", - "score": 2.7623348535297314 - }, - { - "id": "10293", - "score": 2.7623348535297314 - }, - { - "id": "10288", - "score": 2.7623348535297314 - }, - { - "id": "10283", - "score": 2.7623348535297314 - }, - { - "id": "10278", - "score": 2.7623348535297314 - }, - { - "id": "10273", - "score": 2.7623348535297314 - }, - { - "id": "10268", - "score": 2.7623348535297314 - }, - { - "id": "10263", - "score": 2.7623348535297314 - }, - { - "id": "10258", - "score": 2.7623348535297314 - }, - { - "id": "10253", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "aurora-from", - "category": "from", - "relevant": { - "10001": 2 - }, - "lexical": [ - { - "id": "10001", - "score": 1 - } - ], - "semantic": [ - { - "id": "10001", - "score": 0.8873294953955672, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-chat", - "category": "chat", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10223", - "score": 0.8813816418287496, - "lexicalRank": 15 - }, - { - "id": "10263", - "score": 0.8749284164561189, - "lexicalRank": 7 - }, - { - "id": "10233", - "score": 0.8722181190572437, - "lexicalRank": 13 - }, - { - "id": "10268", - "score": 0.8715306928342087, - "lexicalRank": 6 - }, - { - "id": "10218", - "score": 0.8713968455797609, - "lexicalRank": 16 - }, - { - "id": "10208", - "score": 0.8706402050461867, - "lexicalRank": 18 - }, - { - "id": "10253", - "score": 0.870482138126604, - "lexicalRank": 9 - }, - { - "id": "10283", - "score": 0.8698741900246865, - "lexicalRank": 3 - }, - { - "id": "10298", - "score": 0.8697392715074506, - "lexicalRank": 0 - }, - { - "id": "10238", - "score": 0.8696619263428077, - "lexicalRank": 12 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-date", - "category": "date", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10001", - "score": 1 - }, - { - "id": "10002", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10001", - "score": 0.8692216945144376, - "lexicalRank": 0 - }, - { - "id": "10002", - "score": 0.8453639218948915, - "lexicalRank": 1 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-and", - "category": "and", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "aurora-or", - "category": "or", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10298", - "score": 7.489838832938161 - }, - { - "id": "10293", - "score": 7.489838832938161 - }, - { - "id": "10288", - "score": 7.489838832938161 - }, - { - "id": "10283", - "score": 7.489838832938161 - }, - { - "id": "10278", - "score": 7.489838832938161 - }, - { - "id": "10273", - "score": 7.489838832938161 - }, - { - "id": "10268", - "score": 7.489838832938161 - }, - { - "id": "10263", - "score": 7.489838832938161 - }, - { - "id": "10258", - "score": 7.489838832938161 - }, - { - "id": "10253", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "aurora-paraphrase", - "category": "paraphrase", - "relevant": { - "10001": 2, - "10002": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "aurora-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-words", - "category": "words", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10548", - "score": 0.9294003753598464, - "lexicalRank": 253 - }, - { - "id": "10405", - "score": 0.9268079878355586, - "lexicalRank": 39 - }, - { - "id": "10538", - "score": 0.9263857706471276, - "lexicalRank": 255 - }, - { - "id": "10482", - "score": 0.9258707682500287, - "lexicalRank": 146 - }, - { - "id": "10602", - "score": 0.9256494511241109, - "lexicalRank": 122 - }, - { - "id": "10585", - "score": 0.9254712731778304, - "lexicalRank": 3 - }, - { - "id": "10545", - "score": 0.9251230608354597, - "lexicalRank": 11 - }, - { - "id": "10498", - "score": 0.9250750095024941, - "lexicalRank": 263 - }, - { - "id": "10488", - "score": 0.9249750012172294, - "lexicalRank": 265 - }, - { - "id": "10582", - "score": 0.9248633134532314, - "lexicalRank": 126 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-typo", - "category": "typo", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10367", - "score": 0.8806786905837405, - "lexicalRank": 169 - }, - { - "id": "10457", - "score": 0.8802690880138514, - "lexicalRank": 151 - }, - { - "id": "10482", - "score": 0.8795885420606875, - "lexicalRank": 146 - }, - { - "id": "10427", - "score": 0.8788493894056418, - "lexicalRank": 157 - }, - { - "id": "10562", - "score": 0.8784239132925827, - "lexicalRank": 130 - }, - { - "id": "10437", - "score": 0.8777151372157757, - "lexicalRank": 155 - }, - { - "id": "10422", - "score": 0.8773499629723807, - "lexicalRank": 158 - }, - { - "id": "10362", - "score": 0.8771639012061407, - "lexicalRank": 170 - }, - { - "id": "10537", - "score": 0.8769370254072435, - "lexicalRank": 135 - }, - { - "id": "10602", - "score": 0.8767288407478534, - "lexicalRank": 122 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-beginnings", - "category": "beginnings", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10405", - "score": 0.9198652498038274, - "lexicalRank": 39 - }, - { - "id": "10520", - "score": 0.9187540354650561, - "lexicalRank": 16 - }, - { - "id": "10482", - "score": 0.9185876210656136, - "lexicalRank": 146 - }, - { - "id": "10430", - "score": 0.9185690620631375, - "lexicalRank": 34 - }, - { - "id": "10445", - "score": 0.918156502942747, - "lexicalRank": 31 - }, - { - "id": "10602", - "score": 0.9180029649698667, - "lexicalRank": 122 - }, - { - "id": "10427", - "score": 0.917539873668179, - "lexicalRank": 157 - }, - { - "id": "10545", - "score": 0.91736147660192, - "lexicalRank": 11 - }, - { - "id": "10367", - "score": 0.9172398487594343, - "lexicalRank": 169 - }, - { - "id": "10562", - "score": 0.9170926519049367, - "lexicalRank": 130 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-phrase", - "category": "phrase", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 2.7623348535297314 - }, - { - "id": "10595", - "score": 2.7623348535297314 - }, - { - "id": "10590", - "score": 2.7623348535297314 - }, - { - "id": "10585", - "score": 2.7623348535297314 - }, - { - "id": "10580", - "score": 2.7623348535297314 - }, - { - "id": "10575", - "score": 2.7623348535297314 - }, - { - "id": "10570", - "score": 2.7623348535297314 - }, - { - "id": "10565", - "score": 2.7623348535297314 - }, - { - "id": "10560", - "score": 2.7623348535297314 - }, - { - "id": "10555", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "10600", - "score": 2.7623348535297314 - }, - { - "id": "10595", - "score": 2.7623348535297314 - }, - { - "id": "10590", - "score": 2.7623348535297314 - }, - { - "id": "10585", - "score": 2.7623348535297314 - }, - { - "id": "10580", - "score": 2.7623348535297314 - }, - { - "id": "10575", - "score": 2.7623348535297314 - }, - { - "id": "10570", - "score": 2.7623348535297314 - }, - { - "id": "10565", - "score": 2.7623348535297314 - }, - { - "id": "10560", - "score": 2.7623348535297314 - }, - { - "id": "10555", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "cedar-from", - "category": "from", - "relevant": { - "10303": 2 - }, - "lexical": [ - { - "id": "10303", - "score": 1 - } - ], - "semantic": [ - { - "id": "10303", - "score": 0.8648644005237613, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-chat", - "category": "chat", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10510", - "score": 0.8748864283482839, - "lexicalRank": 18 - }, - { - "id": "10595", - "score": 0.8689824008983377, - "lexicalRank": 1 - }, - { - "id": "10525", - "score": 0.8687169265199, - "lexicalRank": 15 - }, - { - "id": "10515", - "score": 0.8684185263437489, - "lexicalRank": 17 - }, - { - "id": "10600", - "score": 0.8659321909233304, - "lexicalRank": 0 - }, - { - "id": "10585", - "score": 0.8658538804618805, - "lexicalRank": 3 - }, - { - "id": "10565", - "score": 0.8651923027273853, - "lexicalRank": 7 - }, - { - "id": "10532", - "score": 0.8649263757798116, - "lexicalRank": 56 - }, - { - "id": "10575", - "score": 0.8646407298812999, - "lexicalRank": 5 - }, - { - "id": "10545", - "score": 0.8641223806525056, - "lexicalRank": 11 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-date", - "category": "date", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10303", - "score": 1 - }, - { - "id": "10304", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10303", - "score": 0.8619150436537566, - "lexicalRank": 0 - }, - { - "id": "10304", - "score": 0.8524625524012509, - "lexicalRank": 1 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-and", - "category": "and", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "cedar-or", - "category": "or", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10600", - "score": 7.489838832938161 - }, - { - "id": "10595", - "score": 7.489838832938161 - }, - { - "id": "10590", - "score": 7.489838832938161 - }, - { - "id": "10585", - "score": 7.489838832938161 - }, - { - "id": "10580", - "score": 7.489838832938161 - }, - { - "id": "10575", - "score": 7.489838832938161 - }, - { - "id": "10570", - "score": 7.489838832938161 - }, - { - "id": "10565", - "score": 7.489838832938161 - }, - { - "id": "10560", - "score": 7.489838832938161 - }, - { - "id": "10555", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "cedar-paraphrase", - "category": "paraphrase", - "relevant": { - "10303": 2, - "10304": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "cedar-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-words", - "category": "words", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10879", - "score": 0.9213124069431595, - "lexicalRank": 127 - }, - { - "id": "10844", - "score": 0.9207854444827525, - "lexicalRank": 134 - }, - { - "id": "10669", - "score": 0.9199201302353809, - "lexicalRank": 169 - }, - { - "id": "10789", - "score": 0.9198138390635751, - "lexicalRank": 145 - }, - { - "id": "10735", - "score": 0.9190050792249866, - "lexicalRank": 276 - }, - { - "id": "10720", - "score": 0.9188604688831258, - "lexicalRank": 279 - }, - { - "id": "10744", - "score": 0.9186825197728935, - "lexicalRank": 154 - }, - { - "id": "10645", - "score": 0.9186674303701673, - "lexicalRank": 294 - }, - { - "id": "10745", - "score": 0.9185491222623472, - "lexicalRank": 274 - }, - { - "id": "10895", - "score": 0.9182719048183844, - "lexicalRank": 244 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-typo", - "category": "typo", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10879", - "score": 0.8985129235259155, - "lexicalRank": 127 - }, - { - "id": "10862", - "score": 0.8962386614132065, - "lexicalRank": 10 - }, - { - "id": "10844", - "score": 0.8955199090239682, - "lexicalRank": 134 - }, - { - "id": "10883", - "score": 0.8951193114737055, - "lexicalRank": 66 - }, - { - "id": "10884", - "score": 0.8948447680739473, - "lexicalRank": 126 - }, - { - "id": "10669", - "score": 0.894776110471853, - "lexicalRank": 169 - }, - { - "id": "10859", - "score": 0.8947304694857922, - "lexicalRank": 131 - }, - { - "id": "10744", - "score": 0.8945010378302453, - "lexicalRank": 154 - }, - { - "id": "10899", - "score": 0.8943928399826122, - "lexicalRank": 123 - }, - { - "id": "10789", - "score": 0.8942916342327378, - "lexicalRank": 145 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-beginnings", - "category": "beginnings", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10879", - "score": 0.8972254388273925, - "lexicalRank": 127 - }, - { - "id": "10674", - "score": 0.8958923841763743, - "lexicalRank": 168 - }, - { - "id": "10789", - "score": 0.8956781566306027, - "lexicalRank": 145 - }, - { - "id": "10844", - "score": 0.8956174612001324, - "lexicalRank": 134 - }, - { - "id": "10744", - "score": 0.8955839035777104, - "lexicalRank": 154 - }, - { - "id": "10659", - "score": 0.8948341665233703, - "lexicalRank": 171 - }, - { - "id": "10669", - "score": 0.8943952104380118, - "lexicalRank": 169 - }, - { - "id": "10709", - "score": 0.8942932783900805, - "lexicalRank": 161 - }, - { - "id": "10799", - "score": 0.8942410899309552, - "lexicalRank": 143 - }, - { - "id": "10884", - "score": 0.8942100783533073, - "lexicalRank": 126 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-phrase", - "category": "phrase", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 3.126652600631594 - }, - { - "id": "10902", - "score": 2.7623348535297314 - }, - { - "id": "10897", - "score": 2.7623348535297314 - }, - { - "id": "10892", - "score": 2.7623348535297314 - }, - { - "id": "10887", - "score": 2.7623348535297314 - }, - { - "id": "10882", - "score": 2.7623348535297314 - }, - { - "id": "10877", - "score": 2.7623348535297314 - }, - { - "id": "10872", - "score": 2.7623348535297314 - }, - { - "id": "10867", - "score": 2.7623348535297314 - }, - { - "id": "10862", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "10606", - "score": 3.126652600631594 - }, - { - "id": "10902", - "score": 2.7623348535297314 - }, - { - "id": "10897", - "score": 2.7623348535297314 - }, - { - "id": "10892", - "score": 2.7623348535297314 - }, - { - "id": "10887", - "score": 2.7623348535297314 - }, - { - "id": "10882", - "score": 2.7623348535297314 - }, - { - "id": "10877", - "score": 2.7623348535297314 - }, - { - "id": "10872", - "score": 2.7623348535297314 - }, - { - "id": "10867", - "score": 2.7623348535297314 - }, - { - "id": "10862", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.27541155237618664, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "север-from", - "category": "from", - "relevant": { - "10605": 2 - }, - "lexical": [ - { - "id": "10605", - "score": 1 - } - ], - "semantic": [ - { - "id": "10605", - "score": 0.8706388952056376, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-chat", - "category": "chat", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10605", - "score": 1 - }, - { - "id": "10606", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10807", - "score": 0.8763211209875281, - "lexicalRank": 21 - }, - { - "id": "10879", - "score": 0.8751973569555798, - "lexicalRank": 47 - }, - { - "id": "10837", - "score": 0.8742873757382752, - "lexicalRank": 15 - }, - { - "id": "10810", - "score": 0.8742405098346124, - "lexicalRank": 101 - }, - { - "id": "10839", - "score": 0.8741222224299756, - "lexicalRank": 55 - }, - { - "id": "10812", - "score": 0.873962415767805, - "lexicalRank": 20 - }, - { - "id": "10899", - "score": 0.8736988678008778, - "lexicalRank": 43 - }, - { - "id": "10865", - "score": 0.8733206746761927, - "lexicalRank": 90 - }, - { - "id": "10859", - "score": 0.8731440824259407, - "lexicalRank": 51 - }, - { - "id": "10842", - "score": 0.8728835335529379, - "lexicalRank": 14 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-date", - "category": "date", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10605", - "score": 1 - }, - { - "id": "10606", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10605", - "score": 0.9013927195872135, - "lexicalRank": 0 - }, - { - "id": "10606", - "score": 0.8751312341989249, - "lexicalRank": 1 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-and", - "category": "and", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "север-or", - "category": "or", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "10606", - "score": 8.47765579013484 - }, - { - "id": "10605", - "score": 8.32129250405688 - }, - { - "id": "10902", - "score": 7.489838832938161 - }, - { - "id": "10897", - "score": 7.489838832938161 - }, - { - "id": "10892", - "score": 7.489838832938161 - }, - { - "id": "10887", - "score": 7.489838832938161 - }, - { - "id": "10882", - "score": 7.489838832938161 - }, - { - "id": "10877", - "score": 7.489838832938161 - }, - { - "id": "10872", - "score": 7.489838832938161 - }, - { - "id": "10867", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "север-paraphrase", - "category": "paraphrase", - "relevant": { - "10605": 2, - "10606": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "север-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-words", - "category": "words", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "11116", - "score": 0.9267224968720181, - "lexicalRank": 140 - }, - { - "id": "11206", - "score": 0.9258146122951406, - "lexicalRank": 122 - }, - { - "id": "10986", - "score": 0.9240344536750787, - "lexicalRank": 166 - }, - { - "id": "10981", - "score": 0.9221301442533495, - "lexicalRank": 167 - }, - { - "id": "11181", - "score": 0.9219162390030617, - "lexicalRank": 127 - }, - { - "id": "11176", - "score": 0.9211905930950566, - "lexicalRank": 128 - }, - { - "id": "11146", - "score": 0.9208713061575157, - "lexicalRank": 134 - }, - { - "id": "11066", - "score": 0.9207864465640064, - "lexicalRank": 150 - }, - { - "id": "11026", - "score": 0.9204614961224764, - "lexicalRank": 158 - }, - { - "id": "11051", - "score": 0.9202586147803542, - "lexicalRank": 153 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-typo", - "category": "typo", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "11116", - "score": 0.9165621307019595, - "lexicalRank": 140 - }, - { - "id": "11206", - "score": 0.9156913304301528, - "lexicalRank": 122 - }, - { - "id": "10986", - "score": 0.9138965186233041, - "lexicalRank": 166 - }, - { - "id": "11176", - "score": 0.9113988597400993, - "lexicalRank": 128 - }, - { - "id": "10981", - "score": 0.9112093128911051, - "lexicalRank": 167 - }, - { - "id": "11181", - "score": 0.9108976500245898, - "lexicalRank": 127 - }, - { - "id": "11066", - "score": 0.9106072424561344, - "lexicalRank": 150 - }, - { - "id": "11051", - "score": 0.9105505198871539, - "lexicalRank": 153 - }, - { - "id": "11146", - "score": 0.9104861584190448, - "lexicalRank": 134 - }, - { - "id": "11142", - "score": 0.9102258582004341, - "lexicalRank": 255 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-beginnings", - "category": "beginnings", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "11206", - "score": 0.9166903870568667, - "lexicalRank": 122 - }, - { - "id": "10986", - "score": 0.9141907260910435, - "lexicalRank": 166 - }, - { - "id": "11116", - "score": 0.914190001936052, - "lexicalRank": 140 - }, - { - "id": "11176", - "score": 0.9122303629510391, - "lexicalRank": 128 - }, - { - "id": "11026", - "score": 0.9122086171694861, - "lexicalRank": 158 - }, - { - "id": "10981", - "score": 0.9121628056814256, - "lexicalRank": 167 - }, - { - "id": "11146", - "score": 0.9121281273747303, - "lexicalRank": 134 - }, - { - "id": "11066", - "score": 0.9121046576597046, - "lexicalRank": 150 - }, - { - "id": "11031", - "score": 0.9119792966738572, - "lexicalRank": 157 - }, - { - "id": "11076", - "score": 0.9116270432881954, - "lexicalRank": 148 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-phrase", - "category": "phrase", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 2.7623348535297314 - }, - { - "id": "11199", - "score": 2.7623348535297314 - }, - { - "id": "11194", - "score": 2.7623348535297314 - }, - { - "id": "11189", - "score": 2.7623348535297314 - }, - { - "id": "11184", - "score": 2.7623348535297314 - }, - { - "id": "11179", - "score": 2.7623348535297314 - }, - { - "id": "11174", - "score": 2.7623348535297314 - }, - { - "id": "11169", - "score": 2.7623348535297314 - }, - { - "id": "11164", - "score": 2.7623348535297314 - }, - { - "id": "11159", - "score": 2.7623348535297314 - } - ], - "semantic": [ - { - "id": "11204", - "score": 2.7623348535297314 - }, - { - "id": "11199", - "score": 2.7623348535297314 - }, - { - "id": "11194", - "score": 2.7623348535297314 - }, - { - "id": "11189", - "score": 2.7623348535297314 - }, - { - "id": "11184", - "score": 2.7623348535297314 - }, - { - "id": "11179", - "score": 2.7623348535297314 - }, - { - "id": "11174", - "score": 2.7623348535297314 - }, - { - "id": "11169", - "score": 2.7623348535297314 - }, - { - "id": "11164", - "score": 2.7623348535297314 - }, - { - "id": "11159", - "score": 2.7623348535297314 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "lumen-from", - "category": "from", - "relevant": { - "10907": 2 - }, - "lexical": [ - { - "id": "10907", - "score": 1 - } - ], - "semantic": [ - { - "id": "10907", - "score": 0.8844509482138534, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-chat", - "category": "chat", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ], - "semantic": [ - { - "id": "10907", - "score": 0.8739870400246, - "lexicalRank": 21 - }, - { - "id": "11149", - "score": 0.8672188484708652, - "lexicalRank": 11 - }, - { - "id": "11181", - "score": 0.8671707718298383, - "lexicalRank": 47 - }, - { - "id": "11201", - "score": 0.8661931762492886, - "lexicalRank": 43 - }, - { - "id": "11146", - "score": 0.8652178390788425, - "lexicalRank": 54 - }, - { - "id": "11156", - "score": 0.8651391606948476, - "lexicalRank": 52 - }, - { - "id": "11176", - "score": 0.8649251345654683, - "lexicalRank": 48 - }, - { - "id": "11206", - "score": 0.8648163875420569, - "lexicalRank": 42 - }, - { - "id": "11196", - "score": 0.8644069415700817, - "lexicalRank": 44 - }, - { - "id": "11151", - "score": 0.8642576487131611, - "lexicalRank": 53 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0.5, - "mrr": 1, - "ndcgAt10": 0.8262346571285599, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-date", - "category": "date", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "10908", - "score": 1 - }, - { - "id": "10907", - "score": 0.9838709677419355 - } - ], - "semantic": [ - { - "id": "10907", - "score": 0.8694133177629458, - "lexicalRank": 1 - }, - { - "id": "10908", - "score": 0.8560303724815496, - "lexicalRank": 0 - } - ], - "lexicalMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 1, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-and", - "category": "and", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "lumen-or", - "category": "or", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "semantic": [ - { - "id": "11204", - "score": 7.489838832938161 - }, - { - "id": "11199", - "score": 7.489838832938161 - }, - { - "id": "11194", - "score": 7.489838832938161 - }, - { - "id": "11189", - "score": 7.489838832938161 - }, - { - "id": "11184", - "score": 7.489838832938161 - }, - { - "id": "11179", - "score": 7.489838832938161 - }, - { - "id": "11174", - "score": 7.489838832938161 - }, - { - "id": "11169", - "score": 7.489838832938161 - }, - { - "id": "11164", - "score": 7.489838832938161 - }, - { - "id": "11159", - "score": 7.489838832938161 - } - ], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": false - }, - { - "id": "lumen-paraphrase", - "category": "paraphrase", - "relevant": { - "10907": 2, - "10908": 1 - }, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-absent", - "category": "no-answer-absent", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - }, - { - "id": "lumen-near-miss", - "category": "no-answer-near-miss", - "relevant": {}, - "lexical": [], - "semantic": [], - "lexicalMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "semanticMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "expanded": true - } - ], - "questionRows": [ - { - "id": "aurora-question-decision", - "category": "decision", - "relevant": { - "10001": 2, - "10002": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10177", - "score": 0.8743954334334848, - "lexicalRank": 207 - }, - { - "id": "10125", - "score": 0.8742033104489153, - "lexicalRank": 157 - }, - { - "id": "10065", - "score": 0.8740301411239777, - "lexicalRank": 169 - }, - { - "id": "10085", - "score": 0.8736044415926192, - "lexicalRank": 165 - }, - { - "id": "10282", - "score": 0.8734653875377917, - "lexicalRank": 186 - }, - { - "id": "10165", - "score": 0.8732298685541525, - "lexicalRank": 149 - }, - { - "id": "10127", - "score": 0.8730335824366351, - "lexicalRank": 217 - }, - { - "id": "10257", - "score": 0.8727338156186585, - "lexicalRank": 191 - }, - { - "id": "10152", - "score": 0.8727318267187806, - "lexicalRank": 212 - }, - { - "id": "10197", - "score": 0.8725556148157968, - "lexicalRank": 203 - } - ], - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "aurora-question-question", - "category": "question", - "relevant": { - "10001": 2, - "10002": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10279", - "score": 0.9055505730054375, - "lexicalRank": 65 - }, - { - "id": "10259", - "score": 0.9053856622231062, - "lexicalRank": 69 - }, - { - "id": "10189", - "score": 0.9053339695175756, - "lexicalRank": 83 - }, - { - "id": "10239", - "score": 0.9053000598760494, - "lexicalRank": 73 - }, - { - "id": "10199", - "score": 0.9049139536790939, - "lexicalRank": 81 - }, - { - "id": "10164", - "score": 0.9016095617494763, - "lexicalRank": 88 - }, - { - "id": "10204", - "score": 0.9013706164104074, - "lexicalRank": 80 - }, - { - "id": "10214", - "score": 0.9008109120007896, - "lexicalRank": 78 - }, - { - "id": "10294", - "score": 0.9007152019365965, - "lexicalRank": 62 - }, - { - "id": "10224", - "score": 0.9005188968223997, - "lexicalRank": 76 - } - ], - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "aurora-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10165", - "score": 0.8726802210445395, - "lexicalRank": 149 - }, - { - "id": "10085", - "score": 0.8709543606664426, - "lexicalRank": 165 - }, - { - "id": "10130", - "score": 0.8706648612232869, - "lexicalRank": 156 - }, - { - "id": "10065", - "score": 0.8703101048025961, - "lexicalRank": 169 - }, - { - "id": "10110", - "score": 0.86957869880922, - "lexicalRank": 160 - }, - { - "id": "10095", - "score": 0.8694762124215225, - "lexicalRank": 163 - }, - { - "id": "10245", - "score": 0.8693494230668928, - "lexicalRank": 133 - }, - { - "id": "10125", - "score": 0.8691772982774707, - "lexicalRank": 157 - }, - { - "id": "10190", - "score": 0.8691229625304249, - "lexicalRank": 144 - }, - { - "id": "10270", - "score": 0.8690160101262605, - "lexicalRank": 128 - } - ], - "lexical": [ - { - "id": "10298", - "score": 1 - }, - { - "id": "10293", - "score": 0.9838709677419355 - }, - { - "id": "10288", - "score": 0.9682539682539681 - }, - { - "id": "10283", - "score": 0.953125 - }, - { - "id": "10278", - "score": 0.9384615384615385 - }, - { - "id": "10273", - "score": 0.9242424242424243 - }, - { - "id": "10268", - "score": 0.9104477611940298 - }, - { - "id": "10263", - "score": 0.8970588235294118 - }, - { - "id": "10258", - "score": 0.8840579710144928 - }, - { - "id": "10253", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "cedar-question-decision", - "category": "decision", - "relevant": { - "10303": 2, - "10304": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10407", - "score": 0.8921191588753772, - "lexicalRank": 161 - }, - { - "id": "10457", - "score": 0.8913712287362181, - "lexicalRank": 151 - }, - { - "id": "10582", - "score": 0.8912151177976313, - "lexicalRank": 126 - }, - { - "id": "10412", - "score": 0.8909905149008674, - "lexicalRank": 160 - }, - { - "id": "10367", - "score": 0.8901243868500228, - "lexicalRank": 169 - }, - { - "id": "10342", - "score": 0.889620914359146, - "lexicalRank": 174 - }, - { - "id": "10482", - "score": 0.8895100338046731, - "lexicalRank": 146 - }, - { - "id": "10512", - "score": 0.8884118601789093, - "lexicalRank": 140 - }, - { - "id": "10427", - "score": 0.8876010515238454, - "lexicalRank": 157 - }, - { - "id": "10357", - "score": 0.8873435565112464, - "lexicalRank": 171 - } - ], - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "cedar-question-question", - "category": "question", - "relevant": { - "10303": 2, - "10304": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10303", - "score": 0.8997394063896005, - "lexicalRank": 60 - }, - { - "id": "10412", - "score": 0.870363248650828, - "lexicalRank": 160 - }, - { - "id": "10457", - "score": 0.8684160915373113, - "lexicalRank": 151 - }, - { - "id": "10407", - "score": 0.8673949648596698, - "lexicalRank": 161 - }, - { - "id": "10367", - "score": 0.8665641081528737, - "lexicalRank": 169 - }, - { - "id": "10431", - "score": 0.8658399495887587, - "lexicalRank": 95 - }, - { - "id": "10526", - "score": 0.8655811438748423, - "lexicalRank": 76 - }, - { - "id": "10471", - "score": 0.8647088527163593, - "lexicalRank": 87 - }, - { - "id": "10506", - "score": 0.8642371766217116, - "lexicalRank": 80 - }, - { - "id": "10447", - "score": 0.8640630498264111, - "lexicalRank": 153 - } - ], - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "cedar-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10407", - "score": 0.8699333934870884, - "lexicalRank": 161 - }, - { - "id": "10357", - "score": 0.8687598750179746, - "lexicalRank": 171 - }, - { - "id": "10367", - "score": 0.8683958480929811, - "lexicalRank": 169 - }, - { - "id": "10412", - "score": 0.8681930254871126, - "lexicalRank": 160 - }, - { - "id": "10457", - "score": 0.8681691135946581, - "lexicalRank": 151 - }, - { - "id": "10482", - "score": 0.8681629432596382, - "lexicalRank": 146 - }, - { - "id": "10602", - "score": 0.8675038158306415, - "lexicalRank": 122 - }, - { - "id": "10362", - "score": 0.8671538232633393, - "lexicalRank": 170 - }, - { - "id": "10342", - "score": 0.8666654261657587, - "lexicalRank": 174 - }, - { - "id": "10402", - "score": 0.8659104858701784, - "lexicalRank": 162 - } - ], - "lexical": [ - { - "id": "10600", - "score": 1 - }, - { - "id": "10595", - "score": 0.9838709677419355 - }, - { - "id": "10590", - "score": 0.9682539682539681 - }, - { - "id": "10585", - "score": 0.953125 - }, - { - "id": "10580", - "score": 0.9384615384615385 - }, - { - "id": "10575", - "score": 0.9242424242424243 - }, - { - "id": "10570", - "score": 0.9104477611940298 - }, - { - "id": "10565", - "score": 0.8970588235294118 - }, - { - "id": "10560", - "score": 0.8840579710144928 - }, - { - "id": "10555", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "север-question-decision", - "category": "decision", - "relevant": { - "10605": 2, - "10606": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10605", - "score": 0.8900251732962743, - "lexicalRank": 1 - }, - { - "id": "10879", - "score": 0.8735637560863058, - "lexicalRank": 127 - }, - { - "id": "10744", - "score": 0.8729735818123653, - "lexicalRank": 154 - }, - { - "id": "10606", - "score": 0.8727974639516136, - "lexicalRank": 0 - }, - { - "id": "10789", - "score": 0.8724961516351566, - "lexicalRank": 145 - }, - { - "id": "10854", - "score": 0.8721360361721472, - "lexicalRank": 132 - }, - { - "id": "10844", - "score": 0.8721247099894074, - "lexicalRank": 134 - }, - { - "id": "10659", - "score": 0.8716292153677793, - "lexicalRank": 171 - }, - { - "id": "10799", - "score": 0.871430651766806, - "lexicalRank": 143 - }, - { - "id": "10849", - "score": 0.8712351264135738, - "lexicalRank": 133 - } - ], - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "север-question-question", - "category": "question", - "relevant": { - "10605": 2, - "10606": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 1, - "mrr": 1, - "ndcgAt10": 0.7967075809905066, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10605", - "score": 0.891559997434239, - "lexicalRank": 1 - }, - { - "id": "10606", - "score": 0.8685433570269113, - "lexicalRank": 0 - }, - { - "id": "10744", - "score": 0.8646542190680107, - "lexicalRank": 154 - }, - { - "id": "10879", - "score": 0.8640846188885009, - "lexicalRank": 127 - }, - { - "id": "10660", - "score": 0.864000293770232, - "lexicalRank": 291 - }, - { - "id": "10799", - "score": 0.8633917953590933, - "lexicalRank": 143 - }, - { - "id": "10854", - "score": 0.8628477048736447, - "lexicalRank": 132 - }, - { - "id": "10844", - "score": 0.8628242883654791, - "lexicalRank": 134 - }, - { - "id": "10720", - "score": 0.8625328190572694, - "lexicalRank": 279 - }, - { - "id": "10650", - "score": 0.8618473253582783, - "lexicalRank": 293 - } - ], - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "север-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10606", - "score": 0.8688934670502048, - "lexicalRank": 0 - }, - { - "id": "10605", - "score": 0.8630904012969371, - "lexicalRank": 1 - }, - { - "id": "10879", - "score": 0.8544312398339066, - "lexicalRank": 127 - }, - { - "id": "10854", - "score": 0.852517259859595, - "lexicalRank": 132 - }, - { - "id": "10744", - "score": 0.8518251851948647, - "lexicalRank": 154 - }, - { - "id": "10639", - "score": 0.8497193354703592, - "lexicalRank": 175 - }, - { - "id": "10899", - "score": 0.8495133421303337, - "lexicalRank": 123 - }, - { - "id": "10789", - "score": 0.8494706401084311, - "lexicalRank": 145 - }, - { - "id": "10889", - "score": 0.8493093202619589, - "lexicalRank": 125 - }, - { - "id": "10849", - "score": 0.8492858369106064, - "lexicalRank": 133 - } - ], - "lexical": [ - { - "id": "10606", - "score": 1 - }, - { - "id": "10605", - "score": 0.9838709677419355 - }, - { - "id": "10902", - "score": 0.9682539682539681 - }, - { - "id": "10897", - "score": 0.953125 - }, - { - "id": "10892", - "score": 0.9384615384615385 - }, - { - "id": "10887", - "score": 0.9242424242424243 - }, - { - "id": "10882", - "score": 0.9104477611940298 - }, - { - "id": "10877", - "score": 0.8970588235294118 - }, - { - "id": "10872", - "score": 0.8840579710144928 - }, - { - "id": "10867", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "lumen-question-decision", - "category": "decision", - "relevant": { - "10907": 2, - "10908": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10907", - "score": 0.8894495730865635, - "lexicalRank": 61 - }, - { - "id": "10908", - "score": 0.8708438836496035, - "lexicalRank": 60 - }, - { - "id": "11206", - "score": 0.8608495452142796, - "lexicalRank": 122 - }, - { - "id": "11010", - "score": 0.8576054454020773, - "lexicalRank": 101 - }, - { - "id": "10986", - "score": 0.8572175978877122, - "lexicalRank": 166 - }, - { - "id": "11146", - "score": 0.8568726575802139, - "lexicalRank": 134 - }, - { - "id": "11165", - "score": 0.8561342933704561, - "lexicalRank": 70 - }, - { - "id": "11116", - "score": 0.8557647490617573, - "lexicalRank": 140 - }, - { - "id": "11080", - "score": 0.8555912262027077, - "lexicalRank": 87 - }, - { - "id": "11066", - "score": 0.8553978516789799, - "lexicalRank": 150 - } - ], - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "lumen-question-question", - "category": "question", - "relevant": { - "10907": 2, - "10908": 1 - }, - "directMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": 0, - "mrr": 0, - "ndcgAt10": 0, - "falseHits": 0 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "10907", - "score": 0.8786636894499708, - "lexicalRank": 61 - }, - { - "id": "10908", - "score": 0.8569269701977336, - "lexicalRank": 60 - }, - { - "id": "11206", - "score": 0.8514216173795147, - "lexicalRank": 122 - }, - { - "id": "11140", - "score": 0.8503204493298434, - "lexicalRank": 75 - }, - { - "id": "11146", - "score": 0.8496648530020674, - "lexicalRank": 134 - }, - { - "id": "11010", - "score": 0.8495310712219786, - "lexicalRank": 101 - }, - { - "id": "11120", - "score": 0.8485951608616598, - "lexicalRank": 79 - }, - { - "id": "11105", - "score": 0.8477906106928664, - "lexicalRank": 82 - }, - { - "id": "11065", - "score": 0.8472162620563428, - "lexicalRank": 90 - }, - { - "id": "11116", - "score": 0.8470660011285048, - "lexicalRank": 140 - } - ], - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ] - }, - { - "id": "lumen-question-no-answer", - "category": "no-answer", - "relevant": {}, - "directMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 0 - }, - "anchorMetrics": { - "recallAt10": null, - "mrr": null, - "ndcgAt10": null, - "falseHits": 10 - }, - "directCandidateCount": 0, - "anchoredCandidateCount": 300, - "semantic": [ - { - "id": "11206", - "score": 0.8733068842004856, - "lexicalRank": 122 - }, - { - "id": "11146", - "score": 0.871852601191307, - "lexicalRank": 134 - }, - { - "id": "11066", - "score": 0.8713609653818737, - "lexicalRank": 150 - }, - { - "id": "11181", - "score": 0.8705610095113754, - "lexicalRank": 127 - }, - { - "id": "11006", - "score": 0.8692922844813956, - "lexicalRank": 162 - }, - { - "id": "11116", - "score": 0.8692283014040969, - "lexicalRank": 140 - }, - { - "id": "10981", - "score": 0.8691945640873089, - "lexicalRank": 167 - }, - { - "id": "11076", - "score": 0.8687983001696971, - "lexicalRank": 148 - }, - { - "id": "11126", - "score": 0.8687221972857682, - "lexicalRank": 138 - }, - { - "id": "11026", - "score": 0.8683696685646625, - "lexicalRank": 158 - } - ], - "lexical": [ - { - "id": "11204", - "score": 1 - }, - { - "id": "11199", - "score": 0.9838709677419355 - }, - { - "id": "11194", - "score": 0.9682539682539681 - }, - { - "id": "11189", - "score": 0.953125 - }, - { - "id": "11184", - "score": 0.9384615384615385 - }, - { - "id": "11179", - "score": 0.9242424242424243 - }, - { - "id": "11174", - "score": 0.9104477611940298 - }, - { - "id": "11169", - "score": 0.8970588235294118 - }, - { - "id": "11164", - "score": 0.8840579710144928 - }, - { - "id": "11159", - "score": 0.8714285714285714 - } - ] - } - ] -} diff --git a/bench/message-search-quality/semantic.ts b/bench/message-search-quality/semantic.ts deleted file mode 100644 index 54b8ce8cf..000000000 --- a/bench/message-search-quality/semantic.ts +++ /dev/null @@ -1,422 +0,0 @@ -import assert from "node:assert/strict" -import { createHash } from "node:crypto" -import { mkdtempSync, readFileSync, writeFileSync } from "node:fs" -import { cpus, tmpdir } from "node:os" -import { join } from "node:path" -import { performance } from "node:perf_hooks" -import type { Message } from "../../dist/domain/models.js" -import { isTextModelInstalled, openEmbedder, textModelsDirectory } from "../../dist/embeddings/embed.js" -import { textModel } from "../../dist/embeddings/models.js" -import { searchCombined } from "../../dist/services/messages-combined.js" -import { openStore } from "../../dist/store/store.js" -import { openCrossEncoder } from "./crossencoder.ts" - -type Query = { id: string; split: "dev" | "test"; category: string; lucene: string; relevant: Record } -type Question = { - id: string - split: "dev" | "test" - category: string - retrieval: string - text: string - relevant: Record -} -type Seed = { - chats: { id: string; title: string }[] - messages: Pick[] -} -type Ranking = { id: string; score: number } -type Metrics = { recallAt10: number | null; mrr: number | null; ndcgAt10: number | null; falseHits: number } -const digest = (bytes: Buffer) => createHash("sha256").update(bytes).digest("hex") -const originalBytes = readFileSync(new URL("./corpus.json", import.meta.url)) -const questionBytes = readFileSync(new URL("./questions.json", import.meta.url)) -const corpus = JSON.parse(originalBytes.toString()) as Seed & { queries: Query[] } -const supplement = JSON.parse(questionBytes.toString()) as Seed & { queries: Question[] } -const root = mkdtempSync(join(tmpdir(), "semantic-message-quality-")) -const crossEncoderMode = process.argv.includes("--crossencoder") -const floors: (number | null)[] = crossEncoderMode - ? [null, -5, -2, -1, 0, 1, 2, 3, 5, 8] - : [null, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95] -const crossEncoderDirectory = process.env.MESSAGE_RERANKER_DIR -if (crossEncoderMode) - assert.ok(crossEncoderDirectory, "MESSAGE_RERANKER_DIR must name a separately prepared local model") -const directory = textModelsDirectory() -const model = textModel("e5-small") -if (!crossEncoderMode) - assert.ok( - isTextModelInstalled(model, directory), - "Install the pinned model separately before running; this benchmark never downloads it", - ) -if (!crossEncoderMode) - for (const file of model.files) - assert.equal(digest(readFileSync(join(directory, model.id, file.name))), file.sha256, `pinned model ${file.name}`) -process.env.TZ = "UTC" -process.env.MESSAGING_STORE = join(root, "quality.db") -process.env.CLI_COMMON_CACHE_DIR = join(root, "empty-cache") -const store = await openStore({ path: process.env.MESSAGING_STORE }) -const account = { provider: "synthetic", account: "500" } -const depths = [50, 100, 300, 500] -const seeds = [corpus, supplement] -const allMessages = seeds.flatMap((seed) => seed.messages) -const messages = new Map(allMessages.map((m) => [m.id, m])) -const vectorCache = new Map() -const queryCache = new Map() -const modeFiles = [ - "services/messages-combined.js", - "search/lucene/parser.js", - "embeddings/embed.js", - "embeddings/models.js", - "store/sqlite/lucene.js", -] -const frozenInputs = { - crossEncoderImplementationSha256: digest(readFileSync(new URL("./crossencoder.ts", import.meta.url))), - rerankerManifestSha256: digest(readFileSync(new URL("./reranker.json", import.meta.url))), - corpusSha256: digest(originalBytes), - questionsSha256: digest(questionBytes), - runnerSha256: digest(readFileSync(new URL("./semantic.ts", import.meta.url))), - productionSha256: Object.fromEntries( - modeFiles.map((file) => [file, digest(readFileSync(new URL(`../../dist/${file}`, import.meta.url)))]), - ), -} -writeFileSync(join(root, "frozen-inputs.json"), JSON.stringify(frozenInputs, null, 2)) -const metrics = (rankings: Ranking[], relevant: Record): Metrics => { - const ids = rankings.slice(0, 10).map((h) => h.id) - const grades = Object.values(relevant) - if (!grades.length) return { recallAt10: null, mrr: null, ndcgAt10: null, falseHits: ids.length } - const first = ids.findIndex((id) => relevant[id]) - const dcg = (g: number[]) => g.reduce((s, v, i) => s + (2 ** v - 1) / Math.log2(i + 2), 0) - return { - recallAt10: ids.filter((id) => relevant[id]).length / grades.length, - mrr: first < 0 ? 0 : 1 / (first + 1), - ndcgAt10: dcg(ids.map((id) => relevant[id] ?? 0)) / dcg([...grades].sort((a, b) => b - a).slice(0, 10)), - falseHits: 0, - } -} -const summarize = (rows: Metrics[]) => { - const answered = rows.filter((row) => row.recallAt10 !== null) - const avg = (key: "recallAt10" | "mrr" | "ndcgAt10") => - answered.length ? answered.reduce((s, r) => s + (r[key] ?? 0), 0) / answered.length : 0 - return { - answerable: answered.length, - noAnswerQueries: rows.length - answered.length, - recallAt10: avg("recallAt10"), - mrr: avg("mrr"), - ndcgAt10: avg("ndcgAt10"), - noAnswerFalseHits: rows.reduce((s, r) => s + r.falseHits, 0), - } -} -const options = (depth: number) => ({ candidateDepth: depth, rrfK: 60, rerank: "none" as const, chatCap: 0 }) -const pool = async (text: string, depth: number) => { - const found = await searchCombined(store, account, { text, limit: depth, timezone: "UTC" }, {}, options(depth)) - assert.equal(new Set(found.items.map((h) => h.id)).size, found.items.length) - assert.ok(found.items.every((h) => messages.has(h.id))) - return { - hits: found.items.map((h) => ({ id: h.id, score: h.score ?? 0 })), - expanded: found.query.combined.expanded, - truncated: found.hasMore, - } -} -const cosine = (a: Float32Array, b: Float32Array) => { - assert.equal(a.length, b.length) - let score = 0 - for (let i = 0; i < a.length; i++) score += (a[i] ?? 0) * (b[i] ?? 0) - return score -} -let embedder: Awaited> | undefined -let crossEncoder: Awaited> | undefined -const pairCache = new Map() -let pairInferenceMs = 0, - scoredPairs = 0 -let passageInferenceMs = 0, - queryInferenceMs = 0, - embeddedMessages = 0 -const rerank = async (text: string, candidates: Ranking[]) => { - if (crossEncoder) { - const ranked = [] - const start = performance.now() - for (const [lexicalRank, hit] of candidates.entries()) { - const key = JSON.stringify([text, hit.id]) - let score = pairCache.get(key) - if (score === undefined) { - assert.ok(scoredPairs < 20000, "bounded experimental pair budget") - score = await crossEncoder.score(text, messages.get(hit.id)?.text ?? "") - pairCache.set(key, score) - scoredPairs++ - } - ranked.push({ id: hit.id, score, lexicalRank }) - } - pairInferenceMs += performance.now() - start - console.error(`Scored ${scoredPairs} synthetic query-message pairs`) - return ranked.sort((a, b) => b.score - a.score || a.lexicalRank - b.lexicalRank) - } - assert.ok(embedder) - let query = queryCache.get(text) - if (!query) { - const start = performance.now() - const [vector] = await embedder.embed([text], "query") - assert.ok(vector) - query = vector - queryCache.set(text, query) - queryInferenceMs += performance.now() - start - } - const missing = candidates.filter((hit) => !vectorCache.has(hit.id)) - if (missing.length) { - const start = performance.now() - const vectors = await embedder.embed( - missing.map((hit) => messages.get(hit.id)?.text ?? ""), - "passage", - ) - assert.equal(vectors.length, missing.length) - missing.forEach((hit, i) => { - const vector = vectors[i] - assert.ok(vector) - vectorCache.set(hit.id, vector) - }) - passageInferenceMs += performance.now() - start - embeddedMessages += missing.length - console.error(`Embedded ${embeddedMessages} synthetic candidates; no network or model download`) - } - return candidates - .map((hit, rank) => { - const vector = vectorCache.get(hit.id) - assert.ok(vector) - return { id: hit.id, score: cosine(query, vector), lexicalRank: rank } - }) - .sort((a, b) => b.score - a.score || a.lexicalRank - b.lexicalRank) -} -try { - for (const seed of seeds) { - await store.saveChats( - account, - seed.chats.map((c) => ({ ...c, kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 100 })), - ) - for (const chat of seed.chats) - await store.saveMessages( - account, - chat.id, - seed.messages - .filter((m) => m.chatId === chat.id) - .map( - (m): Message => ({ - ...m, - senderName: `Synthetic ${m.senderId}`, - editedAt: null, - outgoing: false, - attachments: [], - replyTo: null, - forwardedFrom: null, - reactions: null, - }), - ), - { via: "synthetic" }, - ) - } - await store.fillSearchIndex({}) - await store.fillStems({}) - const dev = corpus.queries.filter((q) => q.split === "dev") - const candidateRows: (Metrics & { - id: string - category: string - depth: number - relevant: Record - candidateIds: string[] - candidateRecall: number | null - truncated: boolean - })[] = [] - for (const q of dev) { - for (const depth of depths) { - const candidates = await pool(q.lucene, depth) - const labels = Object.keys(q.relevant) - candidateRows.push({ - id: q.id, - category: q.category, - depth, - relevant: q.relevant, - candidateIds: candidates.hits.map((h) => h.id), - candidateRecall: labels.length - ? labels.filter((id) => candidates.hits.some((h) => h.id === id)).length / labels.length - : null, - ...metrics(candidates.hits, q.relevant), - truncated: candidates.truncated, - }) - } - } - const candidateSummary = depths.map((depth) => { - const rows = candidateRows.filter((r) => r.depth === depth && r.candidateRecall !== null) - return { - depth, - candidateRecall: rows.reduce((s, r) => s + (r.candidateRecall ?? 0), 0) / rows.length, - byCategory: Object.fromEntries( - [...new Set(rows.map((r) => r.category))].map((category) => { - const selected = rows.filter((r) => r.category === category) - return [category, selected.reduce((s, r) => s + (r.candidateRecall ?? 0), 0) / selected.length] - }), - ), - } - }) - const maxCandidateRecall = Math.max(...candidateSummary.map((r) => r.candidateRecall)) - const selectedDepth = candidateSummary.find((r) => r.candidateRecall === maxCandidateRecall)?.depth ?? 500 - console.error(`Dev candidate recall measured; selected depth ${selectedDepth} on dev only`) - const loaded = performance.now() - if (crossEncoderMode) { - assert.ok(crossEncoderDirectory) - crossEncoder = await openCrossEncoder(crossEncoderDirectory) - } else embedder = await openEmbedder(model, directory, { threads: 8 }) - const loadMs = performance.now() - loaded - const keywordRows = [] - for (const q of dev) { - const candidates = await pool(q.lucene, selectedDepth) - const semantic = candidates.expanded ? await rerank(q.lucene, candidates.hits) : candidates.hits - keywordRows.push({ - id: q.id, - category: q.category, - relevant: q.relevant, - lexical: candidates.hits.slice(0, 10), - semantic: semantic.slice(0, 10), - lexicalMetrics: metrics(candidates.hits, q.relevant), - semanticMetrics: metrics(semantic, q.relevant), - expanded: candidates.expanded, - }) - } - const questionRows: { - id: string - category: string - relevant: Record - directMetrics: Metrics - anchorMetrics: Metrics - directCandidateCount: number - anchoredCandidateCount: number - semantic: Ranking[] - lexical: Ranking[] - }[] = [] - for (const q of supplement.queries.filter((q) => q.split === "dev")) { - const direct = await pool(q.text, selectedDepth) - const anchored = await pool(q.retrieval, selectedDepth) - const semantic = await rerank(q.text, anchored.hits) - questionRows.push({ - id: q.id, - category: q.category, - relevant: q.relevant, - directMetrics: metrics(direct.hits, q.relevant), - anchorMetrics: metrics(anchored.hits, q.relevant), - directCandidateCount: direct.hits.length, - anchoredCandidateCount: anchored.hits.length, - semantic, - lexical: anchored.hits.slice(0, 10), - }) - } - const keywordSummary = { - lexical: summarize(keywordRows.map((r) => r.lexicalMetrics)), - semantic: summarize(keywordRows.map((r) => r.semanticMetrics)), - } - const questionSweep = floors.map((floor) => ({ - floor, - metrics: summarize( - questionRows.map((r) => - metrics( - r.semantic.filter((h) => floor === null || h.score > floor), - r.relevant, - ), - ), - ), - })) - const passing = (m: ReturnType) => - m.recallAt10 >= 0.45 && m.mrr >= 0.5 && m.ndcgAt10 >= 0.4 && m.noAnswerFalseHits === 0 - const eligible = questionSweep.filter((row) => passing(row.metrics)) - const chosen = [...(eligible.length ? eligible : questionSweep)].sort( - (a, b) => a.metrics.noAnswerFalseHits - b.metrics.noAnswerFalseHits || b.metrics.ndcgAt10 - a.metrics.ndcgAt10, - )[0] - assert.ok(chosen) - const devQualified = passing(keywordSummary.semantic) && eligible.length > 0 - const heldOutKeywordRows: { - id: string - metrics: Metrics - lexicalMetrics: Metrics - relevant: Record - hits: Ranking[] - }[] = [] - const heldOutRows = [] - if (devQualified) { - for (const q of corpus.queries.filter((q) => q.split === "test")) { - const candidates = await pool(q.lucene, selectedDepth) - const ranked = candidates.expanded ? await rerank(q.lucene, candidates.hits) : candidates.hits - heldOutKeywordRows.push({ - id: q.id, - relevant: q.relevant, - hits: ranked.slice(0, 10), - metrics: metrics(ranked, q.relevant), - lexicalMetrics: metrics(candidates.hits, q.relevant), - }) - } - for (const q of supplement.queries.filter((q) => q.split === "test")) { - const candidates = await pool(q.retrieval, selectedDepth) - const ranked = await rerank(q.text, candidates.hits) - const hits = ranked.filter((h) => chosen.floor === null || h.score > chosen.floor) - heldOutRows.push({ id: q.id, relevant: q.relevant, hits: hits.slice(0, 10), metrics: metrics(hits, q.relevant) }) - } - } - console.log( - JSON.stringify( - { - ...frozenInputs, - node: process.version, - cpu: cpus()[0]?.model, - storePath: process.env.MESSAGING_STORE, - model: crossEncoder?.model ?? { - id: model.id, - files: model.files.map(({ name, sha256 }) => ({ name, sha256 })), - threads: 8, - }, - kind: crossEncoderMode - ? "cross-encoder raw logit reranking" - : "embedding-cosine reranking; not a cross-encoder", - candidateSummary, - selectedOnDev: { depth: selectedDepth, questionScoreFloor: chosen.floor }, - keywordSummary, - questionSummary: { - direct: summarize(questionRows.map((r) => r.directMetrics)), - anchoredLexical: summarize(questionRows.map((r) => r.anchorMetrics)), - sweep: questionSweep, - }, - devQualified, - questionRetrieval: - "manual topic anchors for diagnostic reranking; full-question direct retrieval is measured separately", - heldOut: devQualified - ? { - keyword: { - lexical: summarize(heldOutKeywordRows.map((r) => r.lexicalMetrics)), - semantic: summarize(heldOutKeywordRows.map((r) => r.metrics)), - rows: heldOutKeywordRows, - }, - questions: { metrics: summarize(heldOutRows.map((r) => r.metrics)), rows: heldOutRows }, - } - : null, - inference: { - pairInferenceMs, - scoredPairs, - averagePairMs: scoredPairs ? pairInferenceMs / scoredPairs : null, - loadMs, - embeddedMessages, - uniqueEmbeddedQueries: queryCache.size, - uniqueCrossEncoderQueries: new Set( - [...pairCache.keys()].map((key) => (JSON.parse(key) as [string, string])[0]), - ).size, - passageInferenceMs, - queryInferenceMs, - amortizedPerMessageMs: embeddedMessages ? passageInferenceMs / embeddedMessages : null, - cache: crossEncoderMode - ? "exact synthetic query-message score cache, empty at run start" - : "new in-memory synthetic message vectors; no persistent production message index", - }, - candidateRows, - keywordRows, - questionRows: questionRows.map((row) => ({ ...row, semantic: row.semantic.slice(0, 10) })), - }, - null, - 2, - ), - ) -} finally { - await crossEncoder?.close() - await embedder?.close() - await store.close() -} diff --git a/bench/message-search-quality/tsconfig.json b/bench/message-search-quality/tsconfig.json deleted file mode 100644 index e3b79c572..000000000 --- a/bench/message-search-quality/tsconfig.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "extends": "@leemour/cli-core/tsconfig.base.json", - "compilerOptions": { - "noEmit": true, - "composite": false, - "declaration": false, - "declarationMap": false, - "allowImportingTsExtensions": true - }, - "include": [ - "*.ts" - ] -} diff --git a/docs/dev/combined-search-evaluation.md b/docs/dev/combined-search-evaluation.md deleted file mode 100644 index 28bfbd46b..000000000 --- a/docs/dev/combined-search-evaluation.md +++ /dev/null @@ -1,160 +0,0 @@ -# Combined message search: evidence, pipeline and measurement - -The public search is unchanged. The internal lexical service and local reranking experiments are -in draft PR [803](https://github.com/leemour/cli-messaging/pull/803). The -[design](combined-search.md) describes the pending transition; the -[benchmark README](../../bench/message-search-quality/README.md) links reproducible inputs, runners, -model pins and raw results. No experiment opens the real store or a messenger account. - -## What the experiments establish - -| Experiment | Evidence | What it contributes | -|---|---|---| -| Strict and legacy baselines | Strict avoids no-answer noise; legacy recovers spelling variants but fills no-answer pages | Separates matching coverage from noise | -| Independent lexical candidates and 180 ranking configurations | Dev recall 0.3625 with no-answer false hits zero; lexical signal variants do not improve the top ten | Retains useful forgiving retrieval without the broad fallback chain; simple lexical signals are insufficient on these fixtures | -| Candidate pool diagnostics | Dev candidate recall 0.4375/0.7500/0.9000/0.9000 at depths 50/100/300/500 | Most labelled messages can be retrieved before ranking; paraphrases are absent from the lexical pool | -| e5-small embedding cosine | Dev recall falls to 0.275; no question threshold both preserves answers and rejects all missing-fact hits | Rejects this tested embedding similarity as the final message ranker | -| Joint query-message reranker | Improves held-out recall, MRR and nDCG over both baselines, with zero no-answer false hits | Provides measured relevance ranking gains, with an expensive scoring stage | -| Question-form diagnostic | Fresh held-out MRR 1.0 and zero missing-fact false hits using manual topic anchors | Shows the value of explicit task wording; does not implement natural-language candidate retrieval | - -The joint reranker's original held-out keyword results are: - -| Search | Recall@10 | MRR@10 | nDCG@10 | No-answer false hits | -|---|---|---|---|---| -| Strict | 0.2625 | 0.3250 | 0.2192 | 0 | -| Legacy | 0.3375 | 0.4500 | 0.2993 | 80 | -| Combined + joint reranker | 0.4375 | 0.4875 | 0.4049 | 0 | - -Recall increases by 17.5 percentage points over strict and 10.0 points over legacy. The same-archive -baseline controls for adding the supplemental synthetic messages. The reranker misses the agreed -recall 0.45 and MRR 0.50 gates and the latency gate. It is an improvement on this small synthetic -benchmark, not proof of general real-chat quality or a release-ready default. - -## What retrieval, candidates, scoring and reranking mean - -1. Parse the Lucene query and resolve account authority and hard filters. -2. Retrieve possible messages with the word/stem indexes, prefixes and spelling corrections. - These messages are **candidates**: they passed retrieval eligibility, not a relevance judgment. -3. Deduplicate full message locators and merge the independently ranked lists using reciprocal - rank fusion. Its score is computed from list positions, not by a neural model. -4. In the joint-model experiment, supply the query and each candidate message together to the - model. It produces a numeric relevance score for that pair. -5. Sort the same candidates by their model scores and select the first ten. This is **reranking**; - it cannot recover a message absent from the candidate pool. - -The standard architecture is called **retrieve and rerank**, or **two-stage retrieval**. Our first -stage is lexical retrieval. A dense retriever would instead find nearby text vectors. A -**bi-encoder** independently embeds query and message and compares their vectors; a -**cross-encoder** processes the query and message jointly. The architecture and terminology are -explained in the [Sentence Transformers guide](https://sbert.net/examples/sentence_transformer/applications/retrieve_rerank/README.html). - -The tested bi-encoder is the repository's pinned multilingual e5-small, using unit vectors and -cosine similarity. The better tested cross-encoder is -[mmarco-mMiniLMv2-L12-H384-v1](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1), an -Apache-2.0 multilingual model, pinned as a roughly 119 MB quantized ONNX file plus its tokenizer -and configuration. Its configuration specifies identity activation, so our scores are raw logits, -not percentages or calibrated probabilities. Keyword ranking does not apply a score floor; -question diagnostics use a dev-selected raw score floor of 2. - -The current prototype executes one pair at a time on the CPU through the existing WebAssembly -ONNX runtime with eight threads. Its measured pair cost is about 10–12 ms: 300 fresh pairs cost -about 3–4 seconds. Loading the model costs roughly another second. These are separate stages; -the quoted scoring cost does not mean that the SQLite candidate lookup takes 3–4 seconds. -A different query requires new joint pair evaluations even when the same messages were previously -seen. An exact repeated query-message pair can reuse its score; there is no reusable standalone -message vector from this cross-encoder. Batch inference, native runtime and smaller-model options -need measurements before claiming a speed improvement. - -## What 43.8 percent means - -The labels are grade 2 for the actual answer, grade 1 for useful supporting context, and grade 0 -for distractors. Existing binary recall and MRR count grades 1 and 2 as relevant. nDCG distinguishes -them using gains 1 and 3 respectively. - -For each answerable query, recall@10 is the number of relevant messages in the first ten divided -by the number of eligible labelled relevant messages. We average these fractions equally across -queries (**macro average**). Finding one of two labelled messages gives 0.5 regardless of its -position inside the top ten. Finding both gives 1.0; finding neither gives zero. - -The unrounded held-out value is 0.4375. From the existing saved rankings, the 40 answerable queries -break down into 11 retrieving both of two relevant messages, four retrieving their sole eligible -relevant message, five retrieving one of two, and twenty retrieving none: - -```text -(11 × 1 + 4 × 1 + 5 × 0.5 + 20 × 0) / 40 = 0.4375 -``` - -Thus the fraction of queries with at least one useful result is 20/40 = **50 percent**, and the -fraction retrieving all their relevant messages is 15/40 = **37.5 percent**. These are distinct -from macro recall 43.75 percent. They are descriptive calculations from the frozen rankings, not -new inference or a tuned result. - -Ranking within the top ten is already measured: - -- **MRR@10** averages the reciprocal rank of the first relevant result: first place contributes 1, - second 1/2, tenth 1/10, and a miss zero. Here 19 queries have their first relevant hit at rank 1, - one at rank 2, and twenty miss, giving `(19 + 0.5) / 40 = 0.4875`. This is not a success rate or - an average rank. It ignores additional relevant hits after the first. -- **nDCG@10** discounts later positions, gives the actual answer more gain than supporting context, - and normalizes each query against its ideal ordering. The mean is 0.4049. It is a ranking-quality - score, not a percentage of successful queries. -- **No-answer false hits** are evaluated separately on queries with no relevant labels. Zero false - hits on eight original held-out cases does not prove general answerability; the additional missing- - fact cases use a different diagnostic and a dev-selected threshold. - -The actual-answer distinction matters. Useful support ranks first in some cases where the answer -is absent. From the existing frozen keyword rankings: useful-hit Success@1 is 47.5 percent and -Success@10 is 50.0 percent. With grade 2 alone, answer Success@1 is 30.0 percent, answer Success@3 -40.0 percent, answer Success@10 42.5 percent, and answer-only MRR@10 is 0.35625. - -## Standard evaluation terms and proposed additions - -These are **information retrieval (IR) evaluation** metrics. A test collection consists of a -corpus, queries and relevance judgments; the judgments are often called **qrels**. Saved ranked -outputs are **runs**. We use graded judgments, paired comparisons on the same queries, dev-only -parameter selection, held-out validation and ablations. References: -[Stanford IR evaluation](https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-ranked-retrieval-results-1.html), -[ir-measures definitions](https://ir-measur.es/en/latest/measures.html), and -[TREC](https://trec.nist.gov/overview.html). - -The following are proposals for the next report, not new release gates or implemented runner fields: - -| Measure | Question it answers | -|---|---| -| Success@1/3/5/10, also called hit rate | Does at least one relevant message occur within the first k? | -| Grade-2 Success and MRR | How reliably and how early do we retrieve the actual answer? | -| Recall@1/3/5/10 and candidate recall@50/100/300 | How much labelled evidence survives each stage and cutoff? | -| nDCG@3/10 | Are the strongest answers ahead of weaker support and noise? | -| MAP@10 | How early do all relevant messages appear, using binary relevance? | -| Precision@k and returned-result precision | How much of the displayed list is useful? Specify the denominator for short lists. | -| All-relevant@k | Do we retrieve every labelled relevant message? This is a task diagnostic, not our recall definition. | -| No-answer query hit rate and false-hit count | How often does an unanswerable query produce hits, and how many? | -| Per-category and per-language breakdowns | Do aggregate gains hide typo, phrase, filter or Cyrillic regressions? | -| Stage p50/p95 latency and memory | What do lookup, model load and inference each cost, cold and warm? | - -Fixed-denominator P@10 is useful only with context: most queries label just two messages, so even -a perfect top ten cannot exceed 20 percent precision when remaining slots are counted. The four -single-answer sender controls reduce this set's ideal macro P@10 to 19 percent. Returned-result -precision is different and can be 100 percent for a short list; report result count beside it. - -For an agent looking for an answer, proposed headline measures are grade-2 Success@1/3/10, graded -nDCG@10, evidence recall@10, no-answer behavior and latency. Retain supporting-evidence metrics -alongside them rather than silently changing the old definition or relabelling the corpus. - -More metrics alone do not create stronger evidence. There are only four topic families per -original split, with many correlated query variants. A larger fresh held-out set, varied numbers -of relevant messages, paraphrases, harder missing-fact negatives and independent label review are -more valuable than treating forty queries as forty independent samples. Future uncertainty estimates -should resample topic families, and report paired per-query gains/regressions and counts. - -The [model and architecture research](combined-search-model-research.md) compares modern rerankers, -open System One alternatives, candidate cascades and the measured disk/RAM/download costs. - -## Next work - -Explain and settle the measurement definitions before modifying metric runners or tuning ranking. -Then derive the expanded dashboard from saved runs without repeating inference, and cross-check -standard metrics with a conventional evaluator using our explicit relevance/gain/cutoff settings. -Optimize CPU execution separately from retrieval depth: reducing depth from 300 to 50 already -loses labelled candidates before any reranker can help. Ranking changes require a fresh held-out -set; implementation-only speed changes require equivalence checks and separate timing measurements. diff --git a/docs/dev/combined-search-handoff.md b/docs/dev/combined-search-handoff.md index 47add688e..40ce8d758 100644 --- a/docs/dev/combined-search-handoff.md +++ b/docs/dev/combined-search-handoff.md @@ -55,12 +55,12 @@ text; how a saved search records `legacy`; the SDK compatibility promise; and an 0. **Worktree.** `git -C /home/leemour/Projects/AI/cli-messaging worktree add -b feat/combined-search ../cli-messaging-wt-combined-search origin/main`. Other sessions work here at once ([`COORDINATION.md`](COORDINATION.md)); the main checkout is not yours. -1. **A message-level ranking test set, before any code change.** Add `bench/message-search-quality/`: +1. **A message-level ranking test set, before any code change.** Use [cli-testing performance/search](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/README.md): a committed synthetic corpus (no real account, no network) with large noisy chats where many messages contain the query words but few answer the question; typos, word beginnings, Cyrillic and Latin, phrases, filters (`from:`, `chat:`, `date:`); no-answer queries. Label the relevant messages per query; split dev / held-out. Measure recall@10, MRR, nDCG@10 and no-answer false hits for - the current strict search and the current legacy chain. Check: `node bench/message-search-quality/run.ts > bench/message-search-quality/baseline.json` + the current strict search and the current legacy chain. Check: `python3 performance/search/run.py --messaging-root "$SEARCH_MESSAGING_ROOT" run > /tmp/search-baseline-new.json` from cli-testing — passes when both modes produce numbers for every query. 2. **A plan for the owner — `docs/dev/combined-search.md`.** Design, the baseline numbers, the targets, the breaking changes (section 5) and the release order. Stop and ask the owner to approve diff --git a/docs/dev/combined-search-model-research.md b/docs/dev/combined-search-model-research.md deleted file mode 100644 index ccb6c122a..000000000 --- a/docs/dev/combined-search-model-research.md +++ /dev/null @@ -1,172 +0,0 @@ -# Local ranking models and decision layers - -This research supports the [combined-search design](combined-search.md) and -[evaluation notes](combined-search-evaluation.md). It proposes experiments rather than a public -model/default change. The current model is measured locally; alternative model sizes come from -publisher model cards and public Hub file metadata. Published GPU timings are not laptop CPU timings. - -## Ways to use the candidate pool - -The current dev pool contains 90 percent of labelled relevant messages at depth 300; every -non-paraphrase category has full candidate recall and paraphrases have zero. It does not mean that -90 percent of the 300 returned messages are relevant. Most are distractors. Candidate recall is a -coverage measure, not precision, and these pool measurements are on dev, not a claimed universal rate. - -A proposed cascade keeps wide retrieval but spends expensive inference selectively: - -```mermaid -flowchart LR - A[Lucene filters and forgiving retrieval] --> B[Wide candidate pool] - B --> C[Cheap ranking and duplicate grouping] - C --> D[Strong reranking of a smaller shortlist] - D --> E[Answer / support / no-answer decision] - E --> F[Labelled message results] -``` - -The smaller shortlist must be chosen by an improved cheap stage, not merely by truncating the -current order: dev candidate recall drops from 0.90 at 300 to 0.4375 at 50. Measure actual-answer -and evidence recall before/after every stage. A separate candidate-generation experiment can -union lexical hits with semantic hits, including existing conversation chunks mapped back to -messages, while preserving authority and hard filters. This could address the paraphrase misses. -The e5 cosine result rejects it as our final reranker; it does not measure or rule out that -candidate-generation role. Noise and no-answer behavior must be evaluated for the full cascade. Grouping should suppress near-identical boilerplate -without discarding genuinely distinct evidence; simple chat caps alone did not improve this corpus. - -Ranking asks which message is relatively better. An answerability decision asks whether the -requested fact is actually present. Keep those separate. A decision model could classify five to -ten finalists as direct answer, support, missing fact, contradictory/outdated, or needing thread -context. Include an explicit no-answer option; choosing among supplied candidates alone can force -an answer even when every candidate is poor. Thresholds need our own held-out calibration. - -Useful candidates should retain matcher provenance, account/chat authority, corrected spellings -and model/version metadata. Returning only three hits does not itself remove the cost of scoring -300. Adaptive expansion or early stopping requires measured miss rates; a high score on an already -examined hit does not establish that unexamined messages contain no better answer. - -## What Jev contributes - -[TypeSafe's introduction](https://typesafe.ai/blog/introducing-system-one-models-and-jev) describes -shared-state typed decisions and non-autoregressive outputs, with vendor-reported 70–500 ms calls. -[The model documentation](https://docs.typesafe.ai/models) describes a hosted API with shared state -and parallel questions, a 64k combined input budget and a 32k state-plus-longest-question budget. -The introduction specifies up to 255 choice options. No public local weights or model size were -found in those official materials. These timings are not measurements on our 300-message workload. - -Our current cross-encoder already emits a scalar without generating text. The interesting System -One features for this task are shared context, typed evidence decisions and uncertainty handling. -Valid output types do not establish factual correctness. A single forward pass still has to process -the supplied text; long lists can have substantial computation and memory costs. - -Open alternatives with publisher-owned repositories: - -- [Laya](https://github.com/NandhaKishorM/laya): Apache-2.0 typed decision models. Its multilingual - checkpoint uses mmBERT-base, about 322M parameters, with an ONNX option and a TypeScript project. - The multilingual default input limit is 1,024 tokens; the documented extension is 8,192. The - advertised 33 ms figure is a GPU result, not our CPU latency. -- [Kev](https://github.com/jaredpalmer/kev): Apache-2.0 shared-state models based on Qwen, starting - at 0.8B. [Kev-0.8B](https://huggingface.co/jaredpalmer/kev-0.8b) is English-only according to its - model card. Its roughly 43 MB adapter also needs the approximately 1.75 GB BF16 base and a head. - CUDA/MLX benchmarks do not establish performance on our Linux CPU. -- [CLM](https://github.com/Contrastive-LM/CLM): separately caches state/action representations and - exposes direct candidate ranking. The reference uses a Qwen3-8B encoder plus projection heads; - its small head download is not the complete model. This is a useful architecture reference, - but the published reference is not a small CPU-first replacement. - -For a Russian/English local decision experiment, Laya-multilingual is the more appropriate first -candidate than Kev-0.8B. This is a suitability judgment from declared language, footprint and -interface, not a benchmark result proving superior search quality. - -## Ranking model shortlist - -Sizes below are selected weight files, decimal MB/GB, excluding tokenizer/config/runtime unless -explicitly stated. Only the current model's RAM has been measured. Quantization may affect ranking -and must be evaluated; different models' scores or thresholds are not interchangeable. - -| Model | Published footprint | Proposed role and constraints | -|---|---|---| -| Current multilingual mMARCO MiniLMv2 | 118.6 MB int8 ONNX; 135.7 MB with tokenizer/config | Controlled baseline; first test native and batched execution | -| [TinyBERT-L2](https://huggingface.co/cross-encoder/ms-marco-TinyBERT-L2-v2) | 4.4M parameters; 4.52 MB int8 ONNX | Very small English speed control; older model, not a multilingual replacement | -| [MiniLM-L6](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2) | 22.7M; 23.2 MB int8 ONNX | English CPU baseline between TinyBERT and the current multilingual model | -| [GTE multilingual reranker](https://huggingface.co/Alibaba-NLP/gte-multilingual-reranker-base) | About 306M; 612 MB FP16 | Apache-2.0 encoder-only reranker covering Russian and 70+ languages; CPU serving is documented | -| [Qwen3-Reranker-0.6B](https://huggingface.co/Qwen/Qwen3-Reranker-0.6B) | 596M; 1.19 GB BF16 | Apache-2.0 multilingual, instruction-conditioned quality comparator; bigger does not establish faster CPU inference | -| [Jina reranker v3.5](https://huggingface.co/jinaai/jina-reranker-v3.5) | 597M; 1.19 GB BF16; official Q4 GGUF about 397 MB plus projector | Modern multilingual listwise ranking; published speed improvements use A100/FlashAttention; CC BY-NC 4.0 limits default commercial adoption | -| [Laya-multilingual](https://huggingface.co/convaiinnovations/laya-multilingual) | 322M; 644 MB safetensors | Apache-2.0 typed finalist/answerability experiment, not a tested drop-in reranker | -| [Answer.AI ColBERT small](https://huggingface.co/answerdotai/answerai-colbert-small-v1) | 33.4M; about 34 MB int8 ONNX | English late-interaction architecture control with reusable message representations | - -[Mixedbread xsmall](https://huggingface.co/mixedbread-ai/mxbai-rerank-xsmall-v1) is another compact -English comparator: 70.8M parameters, about 87 MB quantized ONNX. The -[FlashRank project](https://github.com/PrithivirajDamodaran/FlashRank) demonstrates CPU use of small -ONNX rerankers; its tiny model is the TinyBERT family, not evidence that every multilingual model -can fit in 4 MB. - -[GLiNER2.5](https://github.com/fastino-ai/GLiNER2) is an additional typed classification/extraction -route: 74M small English and 287M multilingual checkpoints. It could supply evidence-role features, -but its declared task is extraction/classification; message relevance and answerability need validation. - -## Listwise and late-interaction alternatives - -Jina v3.5 jointly ranks lists and shares query/context computation; it is architecturally different -from 300 independent pair passes. Its model card reports 1.22–1.56x improvements over v3 on A100, -not a measured speedup over our model or on our CPU. Evaluate document-order sensitivity, list -length, batch merging and RAM; raw scores can depend on the surrounding candidate list. Its -noncommercial weight license makes it a research comparator rather than the default permissive option. - -ColBERT-style **late interaction** precomputes token-level message representations. A new query is -encoded once; MaxSim matches its token vectors against stored message vectors. This shifts neural -message encoding into archive preparation and can avoid online neural inference for every pair. -It adds index storage, freshness/permission handling and build costs. It is different from the -single-vector e5 cosine experiment, so e5's failure does not rule it out. - -[ModernColBERT](https://huggingface.co/lightonai/GTE-ModernColBERT-v1) is an Apache-2.0 English example, -about 149M parameters and a 150 MB int8 export. -[Liquid LFM2.5-ColBERT-350M](https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M) is a newer efficient -multilingual example with about 353M parameters, but its eleven declared languages omit Russian, -and its license is the LFM Open License. Neither should silently become the Russian default. -A suitable multilingual checkpoint and its index costs need separate evaluation. - -## Current measured disk, RAM and preparation - -[resources.ts](../../bench/message-search-quality/resources.ts) loads the existing pinned model in -an isolated Node process and scores 300 short synthetic messages; no store/account is opened. -[resources.json](../../bench/message-search-quality/resources.json) records the model pins, whole- -process RSS at each stage, high-water resident memory, model load and inference times. - -The measured process starts at about 70 MB RSS, uses about 1.06 GB after load and peaks at about -1.33 GB. Scoring 300 short pairs takes about 3.25 seconds; load takes about 1.27 seconds. These are -lab observations for this runtime and workload, not a model-only allocation breakdown or an upper -bound for larger contexts/batches. The resident footprint includes tokenizer objects, ONNX engine, -weights, worker state and intermediate buffers; disk weight size alone does not predict RAM. - -Reproduce after explicit model preparation: - -```sh -MESSAGE_RERANKER_DIR=/tmp/message-search-reranker-1427fd6 \ - node bench/message-search-quality/resources.ts > bench/message-search-quality/resources.json -``` - -The existing [preparation script](../../bench/message-search-quality/prepare-reranker.py) downloads -only the four pinned files in [the manifest](../../bench/message-search-quality/reranker.json), -verifies SHA-256, writes partial downloads separately, and reuses verified files. It does not clone -all precision variants from the model repository. The offline benchmark never installs a model. -A production model category and explicit installer remain future work. - -Download time depends on transfer rate. For the complete 135.7 MB set, transfer-only estimates -are about 109 seconds at 10 Mbit/s, 54 seconds at 20 Mbit/s and 11 seconds at 100 Mbit/s. Add network -setup, server delays and verification. The original download was not timed. A 1.2 GB checkpoint -would take about eight minutes at 20 Mbit/s before that overhead. RAM for alternative models is -unmeasured; checkpoint precision, runtime, input length, batching and accelerator placement matter. - -## Recommended experiment order - -1. Compare the current weights under native CPU execution and length-grouped batches, with ranking - equivalence, memory and cold/warm latency checks. This isolates implementation cost from model quality. -2. Add a tiny English control and one permissively licensed multilingual reranker, then evaluate a - fast-to-strong cascade. Keep the 300-message retrieval pool until shortlist recall proves equivalent. -3. Test Laya-multilingual on a few finalists for actual answer/support/none and context-needed decisions. - Measure answer-only success, false hits, calibration and the extra end-to-end latency. -4. Investigate a multilingual late-interaction index if recurrent online pair inference remains too costly. - Jina listwise ranking is a useful separately licensed architecture comparator. - -No new alternative weights were installed, no hosted Jev call was made, and no public search behavior -changed during this research. Future selection uses fresh held-out families rather than retuning -against the already exposed keyword/question results. diff --git a/docs/dev/combined-search.md b/docs/dev/combined-search.md index c7e154b07..be0e139b3 100644 --- a/docs/dev/combined-search.md +++ b/docs/dev/combined-search.md @@ -5,14 +5,14 @@ ranking fail the dev quality gates; joint query-message ranking improves the mea CLI/MCP and SDK defaults remain unchanged while latency and the original held-out quality gates remain unmet. Natural-language question retrieval is a separate diagnostic, not a public feature. -The [evaluation notes](combined-search-evaluation.md) record the contribution of each experiment, +The [evaluation notes](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/combined-search-evaluation.md) record the contribution of each experiment, the retrieval/scoring pipeline, metric interpretation and proposed next measurements. ## Evidence and acceptance targets -The [message benchmark](../../bench/message-search-quality/README.md) uses 2,616 synthetic messages, +The [message benchmark](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/message-search/README.md) uses 2,616 synthetic messages, 96 queries, ready word/stem indexes, a temporary store and the current production `searchStore`. -Its [raw baseline](../../bench/message-search-quality/baseline.json) contains every query in both modes. +Its [raw baseline](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/message-search/baseline.json) contains every query in both modes. Four topic families per split give 40 answerable and eight no-answer queries each; there is no real account, model download or network use. Reproduce with the benchmark README's three commands. @@ -54,7 +54,7 @@ can rerank using term strength/coverage, minimum token span and phrase adjacency AST nodes, hydrated body bytes, reranking work and elapsed time; all candidate paths check account scope and cancellation. It is not wired to public services, CLI, MCP, bots or saved replay. -[The dev sweep](../../bench/message-search-quality/combined-dev.json) varies all 180 combinations: +[The dev sweep](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/message-search/combined-dev.json) varies all 180 combinations: depth 50/100/300/500, RRF k 20/60/100, no reranker/coverage/proximity/phrase/all, and chat caps 0/3/5. Every combination produces the same dev relevance metrics: @@ -96,7 +96,7 @@ quality, latency and query-contract checks. ## Semantic experiment -The [semantic benchmark](../../bench/message-search-quality/README.md#candidate-recall-and-semantic-ranking) +The [semantic benchmark](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/message-search/README.md#candidate-recall-and-semantic-ranking) preserves the original corpus/labels and adds separately authored question queries plus fresh held-out families. Candidate recall reaches 0.90 at depth 300: every non-paraphrase category has full candidate recall, while paraphrase requests have none. Most misses in the top ten are ranking @@ -108,7 +108,7 @@ answers while eliminating the new no-answer hits. Reject embedding cosine as thi The benchmark then evaluates an official Apache-2.0 quantized [query-message model](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1), pinned separately -in [reranker.json](../../bench/message-search-quality/reranker.json). It scores query and message +in [reranker.json](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/message-search/reranker.json). It scores query and message together rather than comparing separately generated vectors. The benchmark uses its raw identity logits, with no automatic model installation, and freezes the dev-selected configuration before held-out inference. This is a benchmark adapter, not a public model catalogue or service addition. From 2452271682add554a1e25ab434d5b543226e2514 Mon Sep 17 00:00:00 2001 From: leemour Date: Sat, 10 Oct 2026 15:44:22 +0200 Subject: [PATCH 09/14] feat(search): add bounded archive discovery with eligible direct replies --- CHANGELOG.md | 10 + docs/dev/combined-search-handoff.md | 168 +- docs/dev/combined-search.md | 10 +- docs/dev/search-discovery-plan.md | 28 + docs/search/query-language.md | 22 + .../migration.sql | 1 + .../snapshot.json | 5157 +++++++++++++++++ src/cli/messenger/messages-search-command.ts | 11 +- src/mcp/tools/search.test.ts | 8 + src/mcp/tools/search.ts | 16 +- src/search/question-plan.test.ts | 67 + src/search/question-plan.ts | 106 + src/services/messages-combined.ts | 2 +- src/services/messages-discovery.test.ts | 198 + src/services/messages-discovery.ts | 190 + src/services/messages-search.ts | 9 + src/services/messages.ts | 15 +- src/services/searches.ts | 3 + src/store/sqlite/lucene.ts | 7 +- src/store/sqlite/manifest.ts | 1 + src/store/sqlite/migrations.generated.ts | 6 + src/store/sqlite/schema.ts | 3 + src/store/sqlite/search.ts | 34 +- src/store/store.ts | 2 + 24 files changed, 5934 insertions(+), 140 deletions(-) create mode 100644 docs/dev/search-discovery-plan.md create mode 100644 drizzle/20261010132610_version-29-direct-replies/migration.sql create mode 100644 drizzle/20261010132610_version-29-direct-replies/snapshot.json create mode 100644 src/search/question-plan.test.ts create mode 100644 src/search/question-plan.ts create mode 100644 src/services/messages-discovery.test.ts create mode 100644 src/services/messages-discovery.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index c4ca3290e..abdb404bc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,16 @@ Notable changes to `@wirecat/cli-messaging` (`@leemour/cli-messaging` up to 0.21 version, newest first. Versions follow [semver](https://semver.org/); before `1.0.0` a minor version may break callers, and says how under "Changed — may break callers". `pnpm docs:check` checks the shape of this file. +## Unreleased + +### Added + +- `search messages --discover`, MCP `discover=true` and SDK `SearchQuery.discover` find partial + lexical evidence and eligible direct replies in the local archive without model downloads. + Strict Lucene remains the default; explicit syntax, exact and newest searches stay strict. + Result metadata shows matched/missing terms and bounded candidate truncation; scores are not + answer confidence. Migration 29 adds a derived reply lookup index; message data is unchanged. + ## 0.215.0 — 10.10.2026 ### Changed — may break callers diff --git a/docs/dev/combined-search-handoff.md b/docs/dev/combined-search-handoff.md index 40ce8d758..c99611294 100644 --- a/docs/dev/combined-search-handoff.md +++ b/docs/dev/combined-search-handoff.md @@ -1,130 +1,38 @@ -# Handoff — one message search without a language switch - -The trail, optional, grep by `NEED-604`/`NEED-605`: cli-docs `docs/journal/2026-10-09-pr88-merge-and-landing-deploy.md`. - -## 1. What this is - -cli-messaging is the messenger-neutral half of tg-cli and max-cli ([`CLAUDE.md`](../../CLAUDE.md)). -Message search has two modes: strict Lucene (the default: exact words, field filters, Boolean -operators, ranked by word relevance) and `--language legacy` (a fallback chain: every word → word -beginnings → typo correction → any word → raw substring, each step only when the one before found -nothing). The owner finds the switch confusing and wants **one search**: Lucene syntax plus legacy's -forgiving matching, several matchers run in parallel, results merged and **ranked against the -query**, each hit labelled with how it matched so the agent can judge it — because in large chats -and notes many direct hits are irrelevant. Nothing is built yet. The owner asked for a measured -plan first: a ranking test set, then a plan to approve, then the build. - -## 2. Orient in one call - -Run in a worktree off `origin/main` (section 4, step 0): - -```sh -{ echo "## search.ts — the legacy chain"; sed -n '55,106p' src/search/search.ts - echo "## messages.ts — how a request picks lucene/legacy"; sed -n '74,78p;424,428p;684,690p' src/services/messages.ts - echo "## messages-search.ts — strict search returns no corrections"; sed -n '397,432p' src/services/messages-search.ts - echo "## lucene.ts — BM25 ranking only for unstemmed required words"; sed -n '436,486p' src/store/sqlite/lucene.ts - echo "## notes-search.ts — the fusion pattern to reuse"; sed -n '249,256p;320,340p' src/services/notes-search.ts - echo "## parser.ts — strict refuses ~ and points at legacy"; sed -n '255p' src/search/lucene/parser.ts - echo "## searches.ts — saved searches store their language"; sed -n '82,92p' src/services/searches.ts - echo "## spec — legacy kept for SDK callers until a breaking window"; sed -n '45,51p' docs/search/query-language-spec.md - echo "## bot search runs the chain"; grep -n "search(" src/cli/bot/people.ts | head -3 - echo "## open search work"; gh pr list --search "search in:title" --limit 10 -} > ~/.cache/combined-search-orient.txt 2>&1 -``` - -Then read `~/.cache/combined-search-orient.txt` (about 215 lines). It shows: the five legacy steps and -the `Match` labels they already return; how CLI/MCP default to Lucene and when a RegExp forces legacy; -that `searchLucene` always answers `corrections: []`; that the BM25 rank is computed only when the -query has unstemmed required words; the reciprocal-rank-fusion loop notes search uses; the `~` error -text; how a saved search records `legacy`; the SDK compatibility promise; and any open PR on search. - -## 3. Read in this order (only if the orient output is not enough) - -1. `src/search/search.ts:1-54` — what `everyWord`, `correctWords` and `tagged` do, so the forgiving - matchers can be reused as parallel matchers instead of a chain. -2. `src/store/sqlite/lucene.ts:92-110` and `:486-580` — what `matchQuery` returns and where a score - comes from, so fused lists carry a comparable rank. -3. `src/services/embeddings.ts:700-760` — how conversation search already fuses words and meaning - (`RRF_K`, `fused`), and that vectors exist per conversation chunk, not per message. -4. `bench/search-quality/README.md` — how the existing synthetic, offline benchmark is built and run; - the new message-level set follows the same rules. -5. `src/services/search-all.ts:50-110` — how `search all` merges resources by rank; the result shape - readers and MCP already accept. - -## 4. Do - -0. **Worktree.** `git -C /home/leemour/Projects/AI/cli-messaging worktree add -b feat/combined-search ../cli-messaging-wt-combined-search origin/main`. - Other sessions work here at once ([`COORDINATION.md`](COORDINATION.md)); the main checkout is not yours. -1. **A message-level ranking test set, before any code change.** Use [cli-testing performance/search](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/README.md): - a committed synthetic corpus (no real account, no network) with large noisy chats where many - messages contain the query words but few answer the question; typos, word beginnings, Cyrillic and - Latin, phrases, filters (`from:`, `chat:`, `date:`); no-answer queries. Label the relevant messages - per query; split dev / held-out. Measure recall@10, MRR, nDCG@10 and no-answer false hits for - the current strict search and the current legacy chain. Check: `python3 performance/search/run.py --messaging-root "$SEARCH_MESSAGING_ROOT" run > /tmp/search-baseline-new.json` from cli-testing - — passes when both modes produce numbers for every query. -2. **A plan for the owner — `docs/dev/combined-search.md`.** Design, the baseline numbers, the - targets, the breaking changes (section 5) and the release order. Stop and ask the owner to approve - it; do not build before that. -3. **Build after approval.** Shape the plan leans to: - - one query language — Lucene — and no `--language` option in the CLI or MCP; - - matchers run in parallel for plain words: exact words and stems; word beginnings; corrected - spellings; meaning, when the local model is installed (chunks mapped back to their messages); - - lists merged by reciprocal rank fusion as in `notes-search.ts`, then the top candidates re-ranked - against the whole query (all words present, proximity, phrase), with a cap per chat or - conversation so one large chat cannot fill the page; - - each hit carries `match` (reuse the `Match` labels in `search.ts`) and a score; `corrections` - is filled again; - - `--exact` keeps the current strict behaviour for scripts; filters, `AND`/`OR` and phrases stay exact. - Decisions yours, settle each with the test set: candidate depth and the RRF constant (60 is what - notes and conversations use); which re-rank signals earn their cost; the per-chat cap; whether the - weak steps (any word, raw substring) stay as labelled low-rank candidates or go — the owner wants - the agent to judge labelled candidates and objects to unlabelled noise; whether meaning is in the - first release. Check per step: `pnpm lint && pnpm typecheck && pnpm test:coverage && pnpm docs:check`, - and the bench run beats the baseline on held-out without raising no-answer false hits. -4. **Consumers.** tg-cli and max-cli pin an exact cli-messaging version: release first, then a draft - PR in each that bumps it and removes `--language` from their docs and skills (pattern: cli-messaging - #800 with tg-cli #402 and max-cli #523). Changelog lines under the breaking heading in all three. - -## 5. What bites - -1. **The chain cannot simply be deleted.** `bot search messages` calls it with no language - (`src/cli/bot/people.ts:257`) and `--regex` forces legacy (`src/services/messages.ts:424-428`). - Move them onto the combined search or keep the chain as an internal function, not a user option. -2. **Saved searches store `language: "legacy"`** (`src/services/searches.ts:82-92`). Replaying them - needs a conversion; `src/search/lucene/migration.ts` exists but its own warning says the result - loses prefix, typo and any-word matching — with the combined search those come back, so decide - whether a converted saved search is still a "legacy" record at all. -3. **There are two ranking paths, not one.** Unstemmed required words rank by BM25 over - `message_words` (`rankMatch`, `src/store/sqlite/lucene.ts:439`, `:483`); a stemmed search - ranks by stems through a set computed once and joined LEFT (`:491-492` — joined row by row it - took 420 ms instead of 5 at 100k messages). A query with only filters has no rank at all. The - fused score must not compare these raw scores with each other — fuse by rank position, as notes do. -4. **The SDK promise.** `MessagesService.search` and `searchStore` without a language keep legacy - semantics for old SDK callers (`docs/search/query-language-spec.md:47-51`). Changing that is a - stable-export break: follow [README, "How often, and what may break"](../../README.md#how-often-and-what-may-break). -5. **Meaning vectors are per conversation chunk.** `src/services/embeddings.ts` scores chunks; a - meaning matcher for messages must map a chunk to its messages and not flood the list with one chunk. - The e5-small cosine floor of 0.80 was chosen on conversations only (`bench/search-quality`). -6. **The real store is off limits.** No test or bench may open it; `MESSAGING_STORE` points elsewhere - ([`TESTING.md`](TESTING.md)). A new index or table needs a migration number taken first in - [`COORDINATION.md`](COORDINATION.md). -7. **The error text and docs point at legacy.** `src/search/lucene/parser.ts:255` tells users to add - `--language legacy`; `docs/search/query-language.md` and the generated registry tables - (`src/search/lucene/registry.ts`) describe it. `pnpm docs:check` fails on drift between them. - -## 6. Do not touch - -- `src/replies/**` — cli-messaging #800 (reply audience) is open. -- tg-cli and max-cli `docs/*.md` prose — branch `docs/reader-standards` in both repos rewrites every - user page; it already drops `--language legacy` from the prose. Change only `docs/commands.md` - (generated), skills and the changelog in the consumer PRs. -- `bench/search-quality/` — the conversation benchmark; add a new folder beside it. - -## 7. Check - -```sh -pnpm lint && pnpm typecheck && pnpm test:coverage && pnpm docs:check -``` - -The bench runs outside `pnpm test`, with no network and no real store. In tg-cli and max-cli, try an -unreleased cli-messaging with `bin/try-messaging` in the consumer worktree, never a committed `file:` path. +# Message search handoff + +Message search now has an additive archive discovery path under development in +`src/services/messages-discovery.ts`, shared by CLI `--discover`, MCP `discover=true` and SDK +`SearchQuery.discover`. It uses bounded partial lexical matching and eligible direct replies, +without neural downloads or a full archive snapshot. Ordinary word discovery also reuses the +combined word, prefix and correction candidates in `messages-combined.ts`. + +CLI/MCP strict Lucene defaults and no-language SDK compatibility remain unchanged. The earlier +single-default replacement did not pass its approved quality gates; those gates still apply to +any default switch. The explicit discovery option surfaces partial-match evidence, missing terms +and parent provenance rather than claiming to determine whether an answer exists. + +All experiments live in +[cli-testing performance/search](https://github.com/WireCatLabs/cli-testing/blob/test/search-experiments/performance/search/README.md). +Historical reports and model comparisons remain there. Fresh authored wording reduced the frozen +model-free prototype to 9/16 actual answers in the top ten, exposing retrieval failures. Bounded +partial retrieval recovers the missing evidence, while answer ranking remains a measured limit. +The fresh examples are synthetic and authored during this investigation, not independently labeled +real-world judgments. + +Implementation safeguards: + +- Both parent and reply satisfy account, chat, sender, date and `only` filters; explicit syntax, + AST, exact and newest requests retain strict behavior. +- Retrieval and final scoring retain at most 300 candidates, with at most 100 proposed direct + replies, bounded text and the existing execution deadlines. Truncation remains visible. +- Migration 29 adds a live direct-reply index after + [reservation PR 817](https://github.com/WireCatLabs/cli-messaging/pull/817). + The `only` predicate looks up named message keys rather than scanning every archive row. +- Discovery searches the archive. It does not call a messenger server, install a model or rebuild + conversations as a query side effect. Surrounding context and requested thread packets remain + available through their existing paths. + +Required checks remain lint, typecheck, coverage, docs and isolated consumer adoption. Release the +shared package before pinning it in Telegram/MAX. No real store or `src/replies/**` is part of this +work. See [the integration notes](search-discovery-plan.md) for the current delivery sequence and +[the original default-switch proposal](combined-search.md) for its unfulfilled qualification gates. diff --git a/docs/dev/combined-search.md b/docs/dev/combined-search.md index be0e139b3..e058bd07e 100644 --- a/docs/dev/combined-search.md +++ b/docs/dev/combined-search.md @@ -1,5 +1,13 @@ # One message search: measured design +The implemented integration is an additive `--discover` / MCP `discover=true` / SDK +`SearchQuery.discover` archive option, described in [the handoff](combined-search-handoff.md). +The single-default replacement proposed below remains unqualified by its approved gates. +Strict CLI/MCP behavior and no-language SDK compatibility are preserved. The explicit option +returns partial lexical evidence with visible missing terms and bounded context; it does not +establish answer confidence or qualify a default switch. + + Status: the owner approved the lexical and semantic experiments. Lexical reranking and e5 cosine ranking fail the dev quality gates; joint query-message ranking improves the measured results. CLI/MCP and SDK defaults remain unchanged while latency and the original held-out quality gates @@ -270,7 +278,7 @@ This proposal includes the separately agreed breaking window required by the version, updating generated `docs/commands.md`, skills and breaking changelog entries. Avoid other consumer `docs/*.md` prose owned by `docs/reader-standards`. Verify unreleased builds with `bin/try-messaging` in consumer worktrees; no committed `file:` dependency. Arrange same-day - consumer moves when the breaking shared release lands. No release/publish proceeds while the quality gates are unmet. + consumer moves when the breaking shared release lands. The breaking default switch does not proceed while its quality gates are unmet. No migration number is needed now. If saved conversion or later semantic work requires schema changes, reserve a number first through [coordination](COORDINATION.md#store-migrations). diff --git a/docs/dev/search-discovery-plan.md b/docs/dev/search-discovery-plan.md new file mode 100644 index 000000000..92b5a0ec0 --- /dev/null +++ b/docs/dev/search-discovery-plan.md @@ -0,0 +1,28 @@ +# Search discovery integration + +The owner authorized continued search improvement, dependency releases, consumer integration and +merges on 2026-10-10. The current model-free research path depends on a full archive snapshot and +shared synthetic templates. Fresh wording exposes retrieval failures, so it cannot replace strict +search by default. + +The implementation provides an explicit archive discovery option shared by CLI, MCP and SDK. Strict Lucene remains +the compatibility default. Discovery retrieves partial lexical matches using the existing SQLite +FTS indexes and ranks them without neural downloads or a full in-memory corpus. All hard account, +chat, sender, date and explicit syntax constraints remain enforced. Metadata describes partial +matching; results are relevant evidence, not asserted answers or answer probabilities. + +Delivery sequence: + +1. Preserve fresh-fixture evidence and compare bounded OR candidate retrieval with cheap ranking. +2. Add storage-backed discovery using existing FTS BM25 and bounded direct-reply lookup; no base + schema changes or automatic conversation rebuilds. Test hard filters, deleted records, account + isolation, explicit syntax, cancellation and bounds. +3. Expose the opt-in through shared CLI/MCP/SDK entry points and saved-search behavior. Keep remote + search separate from archive discovery. Update shared guides and generated consumer contracts. +4. Measure the built implementation with fresh and historical synthetic fixtures and increasing + archive sizes. Record failures and whole-process resources. +5. Run required checks, merge passing work, release the shared package, adopt the exact released + version in isolated Telegram/MAX worktrees, check offline CLI/MCP adoption and release tools. + +Use synthetic stores only. No real account actions or message reads are part of this work. Public +strict search defaults and explicit Boolean semantics remain unchanged. diff --git a/docs/search/query-language.md b/docs/search/query-language.md index 2970bc85f..b81f7cc64 100644 --- a/docs/search/query-language.md +++ b/docs/search/query-language.md @@ -24,6 +24,28 @@ StandardSyntaxParser и PrecedenceQueryParser, default AND, default field `text` ` store fetch --all --background`, а `coverage.next` в ответе называет, что скачать. Пустой ответ означает «не найдено в выбранном архиве». Проверяйте `coverage`, `completeness` и готовность индекса. Если word index не готов, выполните `store migrate`: строгий поиск не переходит на substring. +## Если формулировка неизвестна + +Для вопроса обычными словами попробуйте `search messages --discover`. Этот режим ищет частичные +совпадения слов и добавляет сохранённые прямые ответы на найденные сообщения. Модели скачивать +не нужно; поиск идёт только в локальном архиве. Ограничения чата, автора и даты сохраняются. + +```sh +tg search messages 'Во сколько ежедневная выгрузка проекта Маяк?' --discover --chat 101 --json +``` + +В MAX замените `tg` на `max`. В MCP передайте `discover: true` инструменту поиска сообщений; +в SDK — `discover: true` в `SearchQuery`. `query.discovery` показывает использованные запросы, +число кандидатов и ограничение выборки; `items[].discovery.missingTerms` — слова, которых нет в +найденном сообщении или его доступном родителе. Откройте сообщения и проверьте контекст: первое +совпадение может быть вопросом, предложением или устаревшим решением, а не ответом. Оценка +ранжирования не означает вероятность правильного ответа. Перефразирование помогает только там, +где есть совпадающие слова или небольшой список поддерживаемых замен; общего понимания смысла нет. + +Boolean-синтаксис, фразы в кавычках, wildcard, AST, `--exact` и `--newest` сохраняют строгую +семантику. Без `--discover` действует обычный строгий поиск. Режим не совмещается с legacy, +`--regex` или `--backend server`; даже при backend=both он ищет только архив. + ## Быстрый старт Примеры ниже исполняются тестами через реальные shared CLI и MCP на diff --git a/drizzle/20261010132610_version-29-direct-replies/migration.sql b/drizzle/20261010132610_version-29-direct-replies/migration.sql new file mode 100644 index 000000000..a449f4d11 --- /dev/null +++ b/drizzle/20261010132610_version-29-direct-replies/migration.sql @@ -0,0 +1 @@ +CREATE INDEX `messages_by_reply` ON `messages` (`chat_pk`,`reply_to_native_id`) WHERE reply_to_native_id IS NOT NULL AND deleted_at IS NULL; \ No newline at end of file diff --git a/drizzle/20261010132610_version-29-direct-replies/snapshot.json b/drizzle/20261010132610_version-29-direct-replies/snapshot.json new file mode 100644 index 000000000..44b31afdd --- /dev/null +++ b/drizzle/20261010132610_version-29-direct-replies/snapshot.json @@ -0,0 +1,5157 @@ +{ + "version": "7", + "dialect": "sqlite", + "id": "5bfa8435-8e90-4bbb-b13e-150bde2fb45f", + "prevIds": [ + "525c45c7-5512-4df6-8c7a-decbff7e3fe3" + ], + "ddl": [ + { + "name": "account_identities", + "entityType": "tables" + }, + { + "name": "accounts", + "entityType": "tables" + }, + { + "name": "attachment_texts", + "entityType": "tables" + }, + { + "name": "attachments", + "entityType": "tables" + }, + { + "name": "auto_tag_claims", + "entityType": "tables" + }, + { + "name": "chat_members", + "entityType": "tables" + }, + { + "name": "chat_metadata", + "entityType": "tables" + }, + { + "name": "chats", + "entityType": "tables" + }, + { + "name": "chunk_vectors", + "entityType": "tables" + }, + { + "name": "contact_aliases", + "entityType": "tables" + }, + { + "name": "conversation_chunks", + "entityType": "tables" + }, + { + "name": "conversation_messages", + "entityType": "tables" + }, + { + "name": "conversation_state", + "entityType": "tables" + }, + { + "name": "conversations", + "entityType": "tables" + }, + { + "name": "entities", + "entityType": "tables" + }, + { + "name": "fetch_leases", + "entityType": "tables" + }, + { + "name": "identities", + "entityType": "tables" + }, + { + "name": "identity_link_events", + "entityType": "tables" + }, + { + "name": "identity_links", + "entityType": "tables" + }, + { + "name": "identity_revisions", + "entityType": "tables" + }, + { + "name": "knowledge_reminders", + "entityType": "tables" + }, + { + "name": "links", + "entityType": "tables" + }, + { + "name": "member_counts", + "entityType": "tables" + }, + { + "name": "member_stays", + "entityType": "tables" + }, + { + "name": "membership_batch_members", + "entityType": "tables" + }, + { + "name": "membership_batches", + "entityType": "tables" + }, + { + "name": "message_counter_observations", + "entityType": "tables" + }, + { + "name": "message_links", + "entityType": "tables" + }, + { + "name": "message_revisions", + "entityType": "tables" + }, + { + "name": "message_stems_pending", + "entityType": "tables" + }, + { + "name": "messages", + "entityType": "tables" + }, + { + "name": "note_chunks", + "entityType": "tables" + }, + { + "name": "note_folders", + "entityType": "tables" + }, + { + "name": "note_revisions", + "entityType": "tables" + }, + { + "name": "notes", + "entityType": "tables" + }, + { + "name": "owner_targets", + "entityType": "tables" + }, + { + "name": "persons", + "entityType": "tables" + }, + { + "name": "search_index_state", + "entityType": "tables" + }, + { + "name": "searches", + "entityType": "tables" + }, + { + "name": "store_settings", + "entityType": "tables" + }, + { + "name": "sync_ranges", + "entityType": "tables" + }, + { + "name": "sync_state", + "entityType": "tables" + }, + { + "name": "tags", + "entityType": "tables" + }, + { + "name": "tasks", + "entityType": "tables" + }, + { + "name": "transcripts", + "entityType": "tables" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account_pk", + "entityType": "columns", + "table": "account_identities" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "account_identities" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "first_seen_at", + "entityType": "columns", + "table": "account_identities" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "last_messaged_at", + "entityType": "columns", + "table": "account_identities" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "accounts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provider", + "entityType": "columns", + "table": "accounts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "native_id", + "entityType": "columns", + "table": "accounts" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "accounts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "accounts" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "attachment_pk", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "normalized_text", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "origin", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "extractor", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "content_sha256", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "bytes", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "error", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "written_at", + "entityType": "columns", + "table": "attachment_texts" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "message_pk", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "position", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "mime", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "title", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "url", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "size", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "width", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "height", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "real", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "duration", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provider_ref", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "local_path", + "entityType": "columns", + "table": "attachments" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "auto_tag_claims" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "tag", + "entityType": "columns", + "table": "auto_tag_claims" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "algorithm", + "entityType": "columns", + "table": "auto_tag_claims" + }, + { + "type": "real", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "score", + "entityType": "columns", + "table": "auto_tag_claims" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "fields", + "entityType": "columns", + "table": "auto_tag_claims" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "auto_tag_claims" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "chat_members" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "chat_members" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "chat_metadata" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "title", + "entityType": "columns", + "table": "chat_metadata" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "username", + "entityType": "columns", + "table": "chat_metadata" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "description", + "entityType": "columns", + "table": "chat_metadata" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "fetched_at", + "entityType": "columns", + "table": "chat_metadata" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account_pk", + "entityType": "columns", + "table": "chats" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "native_id", + "entityType": "columns", + "table": "chats" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "chats" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "title", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "unread_count", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "last_message_at", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "participants_count", + "entityType": "columns", + "table": "chats" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provider_metadata", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "updated_at", + "entityType": "columns", + "table": "chats" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "username", + "entityType": "columns", + "table": "chats" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "membership_state", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "1", + "generated": null, + "name": "is_searchable", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "0", + "generated": null, + "name": "message_count", + "entityType": "columns", + "table": "chats" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "members_tracked_at", + "entityType": "columns", + "table": "chats" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "model", + "entityType": "columns", + "table": "chunk_vectors" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "content_hash", + "entityType": "columns", + "table": "chunk_vectors" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "dims", + "entityType": "columns", + "table": "chunk_vectors" + }, + { + "type": "blob", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "vector", + "entityType": "columns", + "table": "chunk_vectors" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "chunk_vectors" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account_pk", + "entityType": "columns", + "table": "contact_aliases" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "contact_aliases" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "alias", + "entityType": "columns", + "table": "contact_aliases" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "alias_folded", + "entityType": "columns", + "table": "contact_aliases" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "updated_at", + "entityType": "columns", + "table": "contact_aliases" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "conversation_pk", + "entityType": "columns", + "table": "conversation_chunks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "ordinal", + "entityType": "columns", + "table": "conversation_chunks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "first_message_pk", + "entityType": "columns", + "table": "conversation_chunks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "last_message_pk", + "entityType": "columns", + "table": "conversation_chunks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "content_hash", + "entityType": "columns", + "table": "conversation_chunks" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text_start", + "entityType": "columns", + "table": "conversation_chunks" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text_end", + "entityType": "columns", + "table": "conversation_chunks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "conversation_pk", + "entityType": "columns", + "table": "conversation_messages" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "message_pk", + "entityType": "columns", + "table": "conversation_messages" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "conversation_state" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "enabled_at", + "entityType": "columns", + "table": "conversation_state" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "built_at", + "entityType": "columns", + "table": "conversation_state" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "algorithm_version", + "entityType": "columns", + "table": "conversation_state" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "current_build", + "entityType": "columns", + "table": "conversation_state" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "first_message_pk", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "build", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "first_at", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "last_at", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "message_count", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "built_at", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "algorithm_version", + "entityType": "columns", + "table": "conversations" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "entities" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "entities" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "entities" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "entities" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "fetch_leases" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "anchor", + "entityType": "columns", + "table": "fetch_leases" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "holder", + "entityType": "columns", + "table": "fetch_leases" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "expires_at", + "entityType": "columns", + "table": "fetch_leases" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "identities" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provider", + "entityType": "columns", + "table": "identities" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "native_id", + "entityType": "columns", + "table": "identities" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "username", + "entityType": "columns", + "table": "identities" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "identities" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "is_bot", + "entityType": "columns", + "table": "identities" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "phone_hmac", + "entityType": "columns", + "table": "identities" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provider_metadata", + "entityType": "columns", + "table": "identities" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "first_seen_at", + "entityType": "columns", + "table": "identities" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "updated_at", + "entityType": "columns", + "table": "identities" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "description", + "entityType": "columns", + "table": "identities" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "identity_link_events" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "identity_link_events" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "from_person_pk", + "entityType": "columns", + "table": "identity_link_events" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "to_person_pk", + "entityType": "columns", + "table": "identity_link_events" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "method", + "entityType": "columns", + "table": "identity_link_events" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "at", + "entityType": "columns", + "table": "identity_link_events" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "by", + "entityType": "columns", + "table": "identity_link_events" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "identity_links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "person_pk", + "entityType": "columns", + "table": "identity_links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "method", + "entityType": "columns", + "table": "identity_links" + }, + { + "type": "real", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "confidence", + "entityType": "columns", + "table": "identity_links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "linked_at", + "entityType": "columns", + "table": "identity_links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "linked_by", + "entityType": "columns", + "table": "identity_links" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "identity_revisions" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "identity_revisions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "identity_revisions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "username", + "entityType": "columns", + "table": "identity_revisions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "description", + "entityType": "columns", + "table": "identity_revisions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "marks", + "entityType": "columns", + "table": "identity_revisions" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "captured_at", + "entityType": "columns", + "table": "identity_revisions" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "uid", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account_pk", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "task_id", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "due_at", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "timezone", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "state", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "1", + "generated": null, + "name": "revision", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "lease_until", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "receipt", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "updated_at", + "entityType": "columns", + "table": "knowledge_reminders" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "from_ref", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "to_ref", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "anchor", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "origin", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "target_text", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "target_folded", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "role", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "evidence", + "entityType": "columns", + "table": "links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provenance", + "entityType": "columns", + "table": "links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "1", + "generated": null, + "name": "confirmed", + "entityType": "columns", + "table": "links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "member_counts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "day", + "entityType": "columns", + "table": "member_counts" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "participants", + "entityType": "columns", + "table": "member_counts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "listed", + "entityType": "columns", + "table": "member_counts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "complete", + "entityType": "columns", + "table": "member_counts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "at", + "entityType": "columns", + "table": "member_counts" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "first_seen_at", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "last_seen_at", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "joined_at", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "invited_by_pk", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "role", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "gone_at", + "entityType": "columns", + "table": "member_stays" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "batch_pk", + "entityType": "columns", + "table": "membership_batch_members" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "identity_pk", + "entityType": "columns", + "table": "membership_batch_members" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "stay_pk", + "entityType": "columns", + "table": "membership_batch_members" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "observed_at", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "started_at", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "complete", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "participants", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "listed", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "membership_batches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "message_pk", + "entityType": "columns", + "table": "message_counter_observations" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "counter", + "entityType": "columns", + "table": "message_counter_observations" + }, + { + "type": "real", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "value", + "entityType": "columns", + "table": "message_counter_observations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "observed_at", + "entityType": "columns", + "table": "message_counter_observations" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "message_counter_observations" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "message_pk", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "parent_pk", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "real", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "confidence", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "method", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "version", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "batch", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "build", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "stale_at", + "entityType": "columns", + "table": "message_links" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "message_pk", + "entityType": "columns", + "table": "message_revisions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text", + "entityType": "columns", + "table": "message_revisions" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "edited_at", + "entityType": "columns", + "table": "message_revisions" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "captured_at", + "entityType": "columns", + "table": "message_revisions" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "message_stems_pending" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account_pk", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "native_id", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "thread_native_id", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "sender_identity_pk", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "sender_chat_native_id", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "sender_name", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "sent_at", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "edited_at", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "deleted_at", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "reply_to_native_id", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "reply_to", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "forward", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "outgoing", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "reactions", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "provider_metadata", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "ingested_at", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "ingested_via", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "normalized_text", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "normalizer_version", + "entityType": "columns", + "table": "messages" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "mentions", + "entityType": "columns", + "table": "messages" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "note_pk", + "entityType": "columns", + "table": "note_chunks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "seq", + "entityType": "columns", + "table": "note_chunks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text_start", + "entityType": "columns", + "table": "note_chunks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text_end", + "entityType": "columns", + "table": "note_chunks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "content_hash", + "entityType": "columns", + "table": "note_chunks" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "note_folders" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "note_folders" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "format", + "entityType": "columns", + "table": "note_folders" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pending_path", + "entityType": "columns", + "table": "note_folders" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account_pk", + "entityType": "columns", + "table": "note_folders" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "note_folders" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "note_pk", + "entityType": "columns", + "table": "note_revisions" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text", + "entityType": "columns", + "table": "note_revisions" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "captured_at", + "entityType": "columns", + "table": "note_revisions" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "folder_id", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "path", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "title", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "front_matter", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "content_hash", + "entityType": "columns", + "table": "notes" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "1", + "generated": null, + "name": "revision", + "entityType": "columns", + "table": "notes" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "export_path", + "entityType": "columns", + "table": "notes" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "notes" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "updated_at", + "entityType": "columns", + "table": "notes" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "deleted_at", + "entityType": "columns", + "table": "notes" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "owner_targets" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "reference", + "entityType": "columns", + "table": "owner_targets" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "folder_id", + "entityType": "columns", + "table": "owner_targets" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "folder_path", + "entityType": "columns", + "table": "owner_targets" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "owner_targets" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "persons" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "uid", + "entityType": "columns", + "table": "persons" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "persons" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "0", + "generated": null, + "name": "is_self", + "entityType": "columns", + "table": "persons" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "persons" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "updated_at", + "entityType": "columns", + "table": "persons" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "search_index_state" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "watermark", + "entityType": "columns", + "table": "search_index_state" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "filled_through", + "entityType": "columns", + "table": "search_index_state" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "terms_through", + "entityType": "columns", + "table": "search_index_state" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "normalizer_version", + "entityType": "columns", + "table": "search_index_state" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "built_at", + "entityType": "columns", + "table": "search_index_state" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "analyzer", + "entityType": "columns", + "table": "search_index_state" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "searches" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "name", + "entityType": "columns", + "table": "searches" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "command", + "entityType": "columns", + "table": "searches" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "params", + "entityType": "columns", + "table": "searches" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "language", + "entityType": "columns", + "table": "searches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "version", + "entityType": "columns", + "table": "searches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "fields_version", + "entityType": "columns", + "table": "searches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "searches" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "last_run_at", + "entityType": "columns", + "table": "searches" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "0", + "generated": null, + "name": "runs", + "entityType": "columns", + "table": "searches" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "key", + "entityType": "columns", + "table": "store_settings" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "value", + "entityType": "columns", + "table": "store_settings" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "at", + "entityType": "columns", + "table": "store_settings" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "sync_ranges" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "from_key", + "entityType": "columns", + "table": "sync_ranges" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "to_key", + "entityType": "columns", + "table": "sync_ranges" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account_pk", + "entityType": "columns", + "table": "sync_state" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "key", + "entityType": "columns", + "table": "sync_state" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "value", + "entityType": "columns", + "table": "sync_state" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "at", + "entityType": "columns", + "table": "sync_state" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "pk", + "entityType": "columns", + "table": "tags" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "taggable_type", + "entityType": "columns", + "table": "tags" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "taggable_pk", + "entityType": "columns", + "table": "tags" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "tag", + "entityType": "columns", + "table": "tags" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "tags" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": "1", + "generated": null, + "name": "manual", + "entityType": "columns", + "table": "tags" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "id", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source_kind", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "account", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "group_key", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "kind", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "state", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "reason", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "origin", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "created_at", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "due_at", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "integer", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "closed_at", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "text", + "notNull": false, + "autoincrement": false, + "default": null, + "generated": null, + "name": "closed_by", + "entityType": "columns", + "table": "tasks" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "chat_pk", + "entityType": "columns", + "table": "transcripts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "message_native_id", + "entityType": "columns", + "table": "transcripts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "text", + "entityType": "columns", + "table": "transcripts" + }, + { + "type": "text", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "source", + "entityType": "columns", + "table": "transcripts" + }, + { + "type": "integer", + "notNull": true, + "autoincrement": false, + "default": null, + "generated": null, + "name": "heard_at", + "entityType": "columns", + "table": "transcripts" + }, + { + "columns": [ + "account_pk" + ], + "tableTo": "accounts", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_account_identities_account_pk_accounts_pk_fk", + "entityType": "fks", + "table": "account_identities" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_account_identities_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "account_identities" + }, + { + "columns": [ + "message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_attachments_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "attachments" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_auto_tag_claims_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "auto_tag_claims" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_chat_members_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "chat_members" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_chat_members_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "chat_members" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_chat_metadata_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "chat_metadata" + }, + { + "columns": [ + "account_pk" + ], + "tableTo": "accounts", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_chats_account_pk_accounts_pk_fk", + "entityType": "fks", + "table": "chats" + }, + { + "columns": [ + "account_pk" + ], + "tableTo": "accounts", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_contact_aliases_account_pk_accounts_pk_fk", + "entityType": "fks", + "table": "contact_aliases" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_contact_aliases_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "contact_aliases" + }, + { + "columns": [ + "conversation_pk" + ], + "tableTo": "conversations", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversation_chunks_conversation_pk_conversations_pk_fk", + "entityType": "fks", + "table": "conversation_chunks" + }, + { + "columns": [ + "first_message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversation_chunks_first_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "conversation_chunks" + }, + { + "columns": [ + "last_message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversation_chunks_last_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "conversation_chunks" + }, + { + "columns": [ + "conversation_pk" + ], + "tableTo": "conversations", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversation_messages_conversation_pk_conversations_pk_fk", + "entityType": "fks", + "table": "conversation_messages" + }, + { + "columns": [ + "message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversation_messages_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "conversation_messages" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversation_state_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "conversation_state" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversations_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "conversations" + }, + { + "columns": [ + "first_message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_conversations_first_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "conversations" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_fetch_leases_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "fetch_leases" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_identity_link_events_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "identity_link_events" + }, + { + "columns": [ + "from_person_pk" + ], + "tableTo": "persons", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_identity_link_events_from_person_pk_persons_pk_fk", + "entityType": "fks", + "table": "identity_link_events" + }, + { + "columns": [ + "to_person_pk" + ], + "tableTo": "persons", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_identity_link_events_to_person_pk_persons_pk_fk", + "entityType": "fks", + "table": "identity_link_events" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_identity_links_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "identity_links" + }, + { + "columns": [ + "person_pk" + ], + "tableTo": "persons", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_identity_links_person_pk_persons_pk_fk", + "entityType": "fks", + "table": "identity_links" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_identity_revisions_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "identity_revisions" + }, + { + "columns": [ + "account_pk" + ], + "tableTo": "accounts", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_knowledge_reminders_account_pk_accounts_pk_fk", + "entityType": "fks", + "table": "knowledge_reminders" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_member_counts_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "member_counts" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_member_stays_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "member_stays" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_member_stays_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "member_stays" + }, + { + "columns": [ + "invited_by_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_member_stays_invited_by_pk_identities_pk_fk", + "entityType": "fks", + "table": "member_stays" + }, + { + "columns": [ + "batch_pk" + ], + "tableTo": "membership_batches", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_membership_batch_members_batch_pk_membership_batches_pk_fk", + "entityType": "fks", + "table": "membership_batch_members" + }, + { + "columns": [ + "identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_membership_batch_members_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "membership_batch_members" + }, + { + "columns": [ + "stay_pk" + ], + "tableTo": "member_stays", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_membership_batch_members_stay_pk_member_stays_pk_fk", + "entityType": "fks", + "table": "membership_batch_members" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_membership_batches_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "membership_batches" + }, + { + "columns": [ + "message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_message_counter_observations_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "message_counter_observations" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_message_links_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "message_links" + }, + { + "columns": [ + "message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_message_links_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "message_links" + }, + { + "columns": [ + "parent_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_message_links_parent_pk_messages_pk_fk", + "entityType": "fks", + "table": "message_links" + }, + { + "columns": [ + "message_pk" + ], + "tableTo": "messages", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_message_revisions_message_pk_messages_pk_fk", + "entityType": "fks", + "table": "message_revisions" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_messages_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "messages" + }, + { + "columns": [ + "account_pk" + ], + "tableTo": "accounts", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_messages_account_pk_accounts_pk_fk", + "entityType": "fks", + "table": "messages" + }, + { + "columns": [ + "sender_identity_pk" + ], + "tableTo": "identities", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_messages_sender_identity_pk_identities_pk_fk", + "entityType": "fks", + "table": "messages" + }, + { + "columns": [ + "note_pk" + ], + "tableTo": "notes", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "CASCADE", + "nameExplicit": false, + "name": "fk_note_chunks_note_pk_notes_pk_fk", + "entityType": "fks", + "table": "note_chunks" + }, + { + "columns": [ + "account_pk" + ], + "tableTo": "accounts", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_note_folders_account_pk_accounts_pk_fk", + "entityType": "fks", + "table": "note_folders" + }, + { + "columns": [ + "note_pk" + ], + "tableTo": "notes", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_note_revisions_note_pk_notes_pk_fk", + "entityType": "fks", + "table": "note_revisions" + }, + { + "columns": [ + "folder_id" + ], + "tableTo": "note_folders", + "columnsTo": [ + "id" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_notes_folder_id_note_folders_id_fk", + "entityType": "fks", + "table": "notes" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_sync_ranges_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "sync_ranges" + }, + { + "columns": [ + "account_pk" + ], + "tableTo": "accounts", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_sync_state_account_pk_accounts_pk_fk", + "entityType": "fks", + "table": "sync_state" + }, + { + "columns": [ + "chat_pk" + ], + "tableTo": "chats", + "columnsTo": [ + "pk" + ], + "onUpdate": "NO ACTION", + "onDelete": "NO ACTION", + "nameExplicit": false, + "name": "fk_transcripts_chat_pk_chats_pk_fk", + "entityType": "fks", + "table": "transcripts" + }, + { + "columns": [ + "account_pk", + "identity_pk" + ], + "nameExplicit": false, + "name": "account_identities_pk", + "entityType": "pks", + "table": "account_identities" + }, + { + "columns": [ + "chat_pk", + "tag" + ], + "nameExplicit": false, + "name": "auto_tag_claims_pk", + "entityType": "pks", + "table": "auto_tag_claims" + }, + { + "columns": [ + "chat_pk", + "identity_pk" + ], + "nameExplicit": false, + "name": "chat_members_pk", + "entityType": "pks", + "table": "chat_members" + }, + { + "columns": [ + "model", + "content_hash" + ], + "nameExplicit": false, + "name": "chunk_vectors_pk", + "entityType": "pks", + "table": "chunk_vectors" + }, + { + "columns": [ + "account_pk", + "identity_pk" + ], + "nameExplicit": false, + "name": "contact_aliases_pk", + "entityType": "pks", + "table": "contact_aliases" + }, + { + "columns": [ + "conversation_pk", + "ordinal" + ], + "nameExplicit": false, + "name": "conversation_chunks_pk", + "entityType": "pks", + "table": "conversation_chunks" + }, + { + "columns": [ + "conversation_pk", + "message_pk" + ], + "nameExplicit": false, + "name": "conversation_messages_pk", + "entityType": "pks", + "table": "conversation_messages" + }, + { + "columns": [ + "chat_pk", + "anchor" + ], + "nameExplicit": false, + "name": "fetch_leases_pk", + "entityType": "pks", + "table": "fetch_leases" + }, + { + "columns": [ + "chat_pk", + "day" + ], + "nameExplicit": false, + "name": "member_counts_pk", + "entityType": "pks", + "table": "member_counts" + }, + { + "columns": [ + "batch_pk", + "identity_pk" + ], + "nameExplicit": false, + "name": "membership_batch_members_pk", + "entityType": "pks", + "table": "membership_batch_members" + }, + { + "columns": [ + "message_pk", + "counter" + ], + "nameExplicit": false, + "name": "message_counter_observations_pk", + "entityType": "pks", + "table": "message_counter_observations" + }, + { + "columns": [ + "note_pk", + "seq" + ], + "nameExplicit": false, + "name": "note_chunks_pk", + "entityType": "pks", + "table": "note_chunks" + }, + { + "columns": [ + "chat_pk", + "from_key" + ], + "nameExplicit": false, + "name": "sync_ranges_pk", + "entityType": "pks", + "table": "sync_ranges" + }, + { + "columns": [ + "account_pk", + "key" + ], + "nameExplicit": false, + "name": "sync_state_pk", + "entityType": "pks", + "table": "sync_state" + }, + { + "columns": [ + "chat_pk", + "message_native_id" + ], + "nameExplicit": false, + "name": "transcripts_pk", + "entityType": "pks", + "table": "transcripts" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "accounts_pk", + "table": "accounts", + "entityType": "pks" + }, + { + "columns": [ + "attachment_pk" + ], + "nameExplicit": false, + "name": "attachment_texts_pk", + "table": "attachment_texts", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "attachments_pk", + "table": "attachments", + "entityType": "pks" + }, + { + "columns": [ + "chat_pk" + ], + "nameExplicit": false, + "name": "chat_metadata_pk", + "table": "chat_metadata", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "chats_pk", + "table": "chats", + "entityType": "pks" + }, + { + "columns": [ + "chat_pk" + ], + "nameExplicit": false, + "name": "conversation_state_pk", + "table": "conversation_state", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "conversations_pk", + "table": "conversations", + "entityType": "pks" + }, + { + "columns": [ + "id" + ], + "nameExplicit": false, + "name": "entities_pk", + "table": "entities", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "identities_pk", + "table": "identities", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "identity_link_events_pk", + "table": "identity_link_events", + "entityType": "pks" + }, + { + "columns": [ + "identity_pk" + ], + "nameExplicit": false, + "name": "identity_links_pk", + "table": "identity_links", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "identity_revisions_pk", + "table": "identity_revisions", + "entityType": "pks" + }, + { + "columns": [ + "uid" + ], + "nameExplicit": false, + "name": "knowledge_reminders_pk", + "table": "knowledge_reminders", + "entityType": "pks" + }, + { + "columns": [ + "id" + ], + "nameExplicit": false, + "name": "links_pk", + "table": "links", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "member_stays_pk", + "table": "member_stays", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "membership_batches_pk", + "table": "membership_batches", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "message_stems_pending_pk", + "table": "message_stems_pending", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "messages_pk", + "table": "messages", + "entityType": "pks" + }, + { + "columns": [ + "id" + ], + "nameExplicit": false, + "name": "note_folders_pk", + "table": "note_folders", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "notes_pk", + "table": "notes", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "owner_targets_pk", + "table": "owner_targets", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "persons_pk", + "table": "persons", + "entityType": "pks" + }, + { + "columns": [ + "name" + ], + "nameExplicit": false, + "name": "search_index_state_pk", + "table": "search_index_state", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "searches_pk", + "table": "searches", + "entityType": "pks" + }, + { + "columns": [ + "key" + ], + "nameExplicit": false, + "name": "store_settings_pk", + "table": "store_settings", + "entityType": "pks" + }, + { + "columns": [ + "pk" + ], + "nameExplicit": false, + "name": "tags_pk", + "table": "tags", + "entityType": "pks" + }, + { + "columns": [ + "id" + ], + "nameExplicit": false, + "name": "tasks_pk", + "table": "tasks", + "entityType": "pks" + }, + { + "columns": [ + { + "value": "identity_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "chat_members_by_identity", + "entityType": "indexes", + "table": "chat_members" + }, + { + "columns": [ + { + "value": "account_pk", + "isExpression": false + }, + { + "value": "\"last_message_at\" desc", + "isExpression": true + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "chats_by_recency", + "entityType": "indexes", + "table": "chats" + }, + { + "columns": [ + { + "value": "content_hash", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "conversation_chunks_by_hash", + "entityType": "indexes", + "table": "conversation_chunks" + }, + { + "columns": [ + { + "value": "message_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "conversation_messages_by_message", + "entityType": "indexes", + "table": "conversation_messages" + }, + { + "columns": [ + { + "value": "chat_pk", + "isExpression": false + }, + { + "value": "build", + "isExpression": false + }, + { + "value": "first_at", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "conversations_by_chat", + "entityType": "indexes", + "table": "conversations" + }, + { + "columns": [ + { + "value": "person_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "identity_links_by_person", + "entityType": "indexes", + "table": "identity_links" + }, + { + "columns": [ + { + "value": "identity_pk", + "isExpression": false + }, + { + "value": "captured_at", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "identity_revisions_by_identity", + "entityType": "indexes", + "table": "identity_revisions" + }, + { + "columns": [ + { + "value": "account_pk", + "isExpression": false + }, + { + "value": "state", + "isExpression": false + }, + { + "value": "due_at", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "knowledge_reminders_due", + "entityType": "indexes", + "table": "knowledge_reminders" + }, + { + "columns": [ + { + "value": "from_ref", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "links_from", + "entityType": "indexes", + "table": "links" + }, + { + "columns": [ + { + "value": "to_ref", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "links_to", + "entityType": "indexes", + "table": "links" + }, + { + "columns": [ + { + "value": "target_folded", + "isExpression": false + } + ], + "isUnique": false, + "where": "to_ref IS NULL", + "origin": "manual", + "name": "links_unresolved", + "entityType": "indexes", + "table": "links" + }, + { + "columns": [ + { + "value": "chat_pk", + "isExpression": false + }, + { + "value": "identity_pk", + "isExpression": false + } + ], + "isUnique": true, + "where": "gone_at IS NULL", + "origin": "manual", + "name": "member_stays_open", + "entityType": "indexes", + "table": "member_stays" + }, + { + "columns": [ + { + "value": "identity_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "member_stays_by_identity", + "entityType": "indexes", + "table": "member_stays" + }, + { + "columns": [ + { + "value": "stay_pk", + "isExpression": false + }, + { + "value": "batch_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "membership_members_by_stay", + "entityType": "indexes", + "table": "membership_batch_members" + }, + { + "columns": [ + { + "value": "chat_pk", + "isExpression": false + }, + { + "value": "observed_at", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "membership_batches_by_chat_time", + "entityType": "indexes", + "table": "membership_batches" + }, + { + "columns": [ + { + "value": "message_pk", + "isExpression": false + }, + { + "value": "ifnull(\"parent_pk\", 0)", + "isExpression": true + }, + { + "value": "source", + "isExpression": false + }, + { + "value": "kind", + "isExpression": false + }, + { + "value": "ifnull(\"build\", 0)", + "isExpression": true + } + ], + "isUnique": true, + "where": null, + "origin": "manual", + "name": "message_links_unique", + "entityType": "indexes", + "table": "message_links" + }, + { + "columns": [ + { + "value": "parent_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "message_links_by_parent", + "entityType": "indexes", + "table": "message_links" + }, + { + "columns": [ + { + "value": "chat_pk", + "isExpression": false + }, + { + "value": "build", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "message_links_by_build", + "entityType": "indexes", + "table": "message_links" + }, + { + "columns": [ + { + "value": "message_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "revisions_by_message", + "entityType": "indexes", + "table": "message_revisions" + }, + { + "columns": [ + { + "value": "chat_pk", + "isExpression": false + }, + { + "value": "\"sent_at\" desc", + "isExpression": true + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "messages_by_time", + "entityType": "indexes", + "table": "messages" + }, + { + "columns": [ + { + "value": "account_pk", + "isExpression": false + }, + { + "value": "native_id", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "messages_by_account", + "entityType": "indexes", + "table": "messages" + }, + { + "columns": [ + { + "value": "sender_identity_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "messages_by_sender", + "entityType": "indexes", + "table": "messages" + }, + { + "columns": [ + { + "value": "chat_pk", + "isExpression": false + }, + { + "value": "reply_to_native_id", + "isExpression": false + } + ], + "isUnique": false, + "where": "reply_to_native_id IS NOT NULL AND deleted_at IS NULL", + "origin": "manual", + "name": "messages_by_reply", + "entityType": "indexes", + "table": "messages" + }, + { + "columns": [ + { + "value": "pk", + "isExpression": false + } + ], + "isUnique": false, + "where": "normalized_text IS NULL AND deleted_at IS NULL", + "origin": "manual", + "name": "messages_to_normalize", + "entityType": "indexes", + "table": "messages" + }, + { + "columns": [ + { + "value": "content_hash", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "note_chunks_by_hash", + "entityType": "indexes", + "table": "note_chunks" + }, + { + "columns": [ + { + "value": "note_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "note_revisions_by_note", + "entityType": "indexes", + "table": "note_revisions" + }, + { + "columns": [ + { + "value": "folder_id", + "isExpression": false + }, + { + "value": "path", + "isExpression": false + } + ], + "isUnique": true, + "where": null, + "origin": "manual", + "name": "notes_by_path", + "entityType": "indexes", + "table": "notes" + }, + { + "columns": [ + { + "value": "folder_id", + "isExpression": false + }, + { + "value": "folder_path", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "owner_targets_by_folder", + "entityType": "indexes", + "table": "owner_targets" + }, + { + "columns": [ + { + "value": "command", + "isExpression": false + }, + { + "value": "params", + "isExpression": false + } + ], + "isUnique": true, + "where": "name IS NULL", + "origin": "manual", + "name": "searches_history", + "entityType": "indexes", + "table": "searches" + }, + { + "columns": [ + { + "value": "\"last_run_at\" desc", + "isExpression": true + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "searches_by_last_run", + "entityType": "indexes", + "table": "searches" + }, + { + "columns": [ + { + "value": "tag", + "isExpression": false + }, + { + "value": "taggable_type", + "isExpression": false + }, + { + "value": "taggable_pk", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "tags_by_tag", + "entityType": "indexes", + "table": "tags" + }, + { + "columns": [ + { + "value": "account", + "isExpression": false + }, + { + "value": "source", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "tasks_by_source", + "entityType": "indexes", + "table": "tasks" + }, + { + "columns": [ + { + "value": "account", + "isExpression": false + }, + { + "value": "state", + "isExpression": false + }, + { + "value": "group_key", + "isExpression": false + } + ], + "isUnique": false, + "where": null, + "origin": "manual", + "name": "tasks_by_state", + "entityType": "indexes", + "table": "tasks" + }, + { + "columns": [ + "provider", + "native_id" + ], + "nameExplicit": false, + "name": "accounts_provider_native_id_unique", + "entityType": "uniques", + "table": "accounts" + }, + { + "columns": [ + "message_pk", + "position" + ], + "nameExplicit": false, + "name": "attachments_message_pk_position_unique", + "entityType": "uniques", + "table": "attachments" + }, + { + "columns": [ + "account_pk", + "native_id" + ], + "nameExplicit": false, + "name": "chats_account_pk_native_id_unique", + "entityType": "uniques", + "table": "chats" + }, + { + "columns": [ + "provider", + "native_id" + ], + "nameExplicit": false, + "name": "identities_provider_native_id_unique", + "entityType": "uniques", + "table": "identities" + }, + { + "columns": [ + "chat_pk", + "native_id" + ], + "nameExplicit": false, + "name": "messages_chat_pk_native_id_unique", + "entityType": "uniques", + "table": "messages" + }, + { + "columns": [ + "taggable_type", + "taggable_pk", + "tag" + ], + "nameExplicit": false, + "name": "tags_taggable_type_taggable_pk_tag_unique", + "entityType": "uniques", + "table": "tags" + }, + { + "columns": [ + "account_pk" + ], + "nameExplicit": false, + "name": "note_folders_account_pk_unique", + "entityType": "uniques", + "table": "note_folders" + }, + { + "columns": [ + "id" + ], + "nameExplicit": false, + "name": "notes_id_unique", + "entityType": "uniques", + "table": "notes" + }, + { + "columns": [ + "reference" + ], + "nameExplicit": false, + "name": "owner_targets_reference_unique", + "entityType": "uniques", + "table": "owner_targets" + }, + { + "columns": [ + "uid" + ], + "nameExplicit": false, + "name": "persons_uid_unique", + "entityType": "uniques", + "table": "persons" + }, + { + "columns": [ + "name" + ], + "nameExplicit": false, + "name": "searches_name_unique", + "entityType": "uniques", + "table": "searches" + } + ], + "renames": [] +} \ No newline at end of file diff --git a/src/cli/messenger/messages-search-command.ts b/src/cli/messenger/messages-search-command.ts index a6e90e30d..a16da910b 100644 --- a/src/cli/messenger/messages-search-command.ts +++ b/src/cli/messenger/messages-search-command.ts @@ -61,8 +61,13 @@ export const messagesSearchCommand = (messenger: Messenger, kind: SearchKind = " "bare words and quotes match their exact form only, as exact:word does; text: still matches every form", ) .option("--context ", "messages before and after each hit; 2 in the terminal, 0 otherwise", wholeCount) - if (!mail) + if (!mail) { + command.option( + "--discover", + "find partial lexical matches and eligible replies in the local archive; results are evidence, not confirmed answers", + ) command.option("--language ", "the query language: strict Lucene or legacy discovery", languageOf) + } command.option("--timezone ", "the IANA timezone for calendar date boundaries") if (mail) return command.action(searchAction(messenger, kind)) return command @@ -90,6 +95,7 @@ const searchAction = (messenger: Messenger, kind: SearchKind) => source, regex, language, + discover, timezone, newest, exact, @@ -100,6 +106,7 @@ const searchAction = (messenger: Messenger, kind: SearchKind) => chat?: string source?: string regex?: boolean + discover?: boolean language?: "lucene" | "legacy" timezone?: string newest?: boolean @@ -119,6 +126,7 @@ const searchAction = (messenger: Messenger, kind: SearchKind) => const typed = { ...(chat === undefined ? {} : { chat }), ...(source === undefined ? {} : { source }), + ...(discover === undefined ? {} : { discover }), ...(language === undefined ? {} : { language }), ...(timezone === undefined ? {} : { timezone }), ...(newest ? { newest: true } : {}), @@ -161,6 +169,7 @@ const searchAction = (messenger: Messenger, kind: SearchKind) => signal, language: params.language ?? (pattern ? "legacy" : "lucene"), newest: params.newest === true, + ...(params.discover === undefined ? {} : { discover: params.discover }), ...(params.exact ? { exact: true } : {}), context: params.context ?? (context.format === "pretty" ? 2 : 0), saved: id, diff --git a/src/mcp/tools/search.test.ts b/src/mcp/tools/search.test.ts index a551a77b5..5681aa1dc 100644 --- a/src/mcp/tools/search.test.ts +++ b/src/mcp/tools/search.test.ts @@ -97,6 +97,14 @@ describe.each(["max", "tg"])("the shared MCP search bridge for %s", (command) => expect(search).not.toHaveBeenCalled() }) + it("forwards explicit discovery and rejects a non-boolean value", async () => { + const args = v.parse(messagesSearchInput(messenger), { text: "When is the Helix export?", discover: true }) + const search = vi.fn(async () => found) + await answerMessagesSearch({ search }, args, { limit: 20 }) + expect(search).toHaveBeenCalledWith(expect.objectContaining({ discover: true })) + expect(() => v.parse(messagesSearchInput(messenger), { text: "Helix", discover: "yes" })).toThrow() + }) + it("rejects a language typo in the common schema", () => { expect(() => v.parse(messagesSearchInput(messenger), { text: "x", language: "typo" })).toThrow() }) diff --git a/src/mcp/tools/search.ts b/src/mcp/tools/search.ts index 29b2f037e..d6bc69ae9 100644 --- a/src/mcp/tools/search.ts +++ b/src/mcp/tools/search.ts @@ -12,7 +12,7 @@ import { chatOf, limit } from "../tool.js" const EXACT = "bare words and quotes match their exact form only, as exact:word does; not with an AST" export const MESSAGES_SEARCH_DESCRIPTION = - "Search the local store, and the messenger's server where it can search, using the Lucene 9.12.3 profile, default AND, with strict Boolean matching. Legacy discovery is explicit with language=legacy. Text or a versioned AST, account-scoped filters, calendar timezone, term/body regex and candidate presets use one service. Empty hits still report archive coverage. `saved` runs a saved search (searches_list) or an earlier run (searches_history). With sync_first, first fetch new messages within max_chats (5), sync_time (30s), max_messages (500), under messages.sync-first permission. A failed or bounded refresh keeps local results with stale coverage and refreshed details. thread=true attaches each hit's bounded parent/reply graph, with provenance and stale-edge labels; thread_hops, thread_messages, thread_bytes, thread_within set its separate bounds. Guide: https://github.com/WireCatLabs/cli-messaging/blob/main/docs/search/query-language.md. Text words and quoted phrases match every form of their words (Snowball stems, exact forms ranked first, query.stemming says how); exact:word, or exact=true, matches the exact form only. Where the messenger's server can search, it is also asked by default (backend=both; archive for the local store only) within server_time (5s), under messages.server-search: its hits are saved and re-checked by the same strict query, each hit says its source (archive, server, both), and server says what the server step did. coverage says what the archive held: messages and chats searched, chats never fetched or behind, up to ten attention chats, and next — the command that would improve the answer; when next is set and nothing was found, run it (or ask the owner) before concluding the message does not exist. Returns { items, page, limit, hasMore, corrections, completeness, wordsReady, stemsReady, query, coverage, server? }." + "Use discover=true for partial lexical evidence and eligible replies to natural questions in the local archive, without model downloads; inspect each hit’s discovery.missingTerms and surrounding messages before answering. Explicit syntax stays strict, and ranking is not answer confidence. Search the local store, and the messenger's server where it can search, using the Lucene 9.12.3 profile, default AND, with strict Boolean matching. Legacy discovery is explicit with language=legacy. Text or a versioned AST, account-scoped filters, calendar timezone, term/body regex and candidate presets use one service. Empty hits still report archive coverage. `saved` runs a saved search (searches_list) or an earlier run (searches_history). With sync_first, first fetch new messages within max_chats (5), sync_time (30s), max_messages (500), under messages.sync-first permission. A failed or bounded refresh keeps local results with stale coverage and refreshed details. thread=true attaches each hit's bounded parent/reply graph, with provenance and stale-edge labels; thread_hops, thread_messages, thread_bytes, thread_within set its separate bounds. Guide: https://github.com/WireCatLabs/cli-messaging/blob/main/docs/search/query-language.md. Text words and quoted phrases match every form of their words (Snowball stems, exact forms ranked first, query.stemming says how); exact:word, or exact=true, matches the exact form only. Where the messenger's server can search, it is also asked by default (backend=both; archive for the local store only) within server_time (5s), under messages.server-search: its hits are saved and re-checked by the same strict query, each hit says its source (archive, server, both), and server says what the server step did. coverage says what the archive held: messages and chats searched, chats never fetched or behind, up to ten attention chats, and next — the command that would improve the answer; when next is set and nothing was found, run it (or ask the owner) before concluding the message does not exist. Returns { items, page, limit, hasMore, corrections, completeness, wordsReady, stemsReady, query, coverage, server? }." const backendInputs = { backend: v.optional( @@ -41,6 +41,14 @@ export const messagesSearchInput = (messenger: Messenger) => ...syncInputs, text: v.optional(v.pipe(v.string(), v.description("the query: Lucene text or explicit legacy syntax"))), ast: v.optional(v.unknown()), + discover: v.optional( + v.pipe( + v.boolean(), + v.description( + "partial lexical discovery with eligible replies in the local archive; use for questions when wording is uncertain; hits are evidence, not confirmed answers", + ), + ), + ), language: v.optional(v.picklist(["lucene", "legacy"])), timezone: v.optional(v.string()), chat: v.optional(chatOf(messenger)), @@ -109,8 +117,8 @@ export type MessagesSearchArgs = v.InferOutput): SearchParams => Object.fromEntries( - ["text", "language", "timezone", "chat", "source", "newest", "exact", "context", "limit", "by"].flatMap((key) => - args[key] === undefined ? [] : [[key, args[key]]], + ["text", "discover", "language", "timezone", "chat", "source", "newest", "exact", "context", "limit", "by"].flatMap( + (key) => (args[key] === undefined ? [] : [[key, args[key]]]), ), ) @@ -152,6 +160,7 @@ export const answerMessagesSearch = async ( ...threadArgs(args), ...(pattern ? { pattern } : params.text === undefined ? {} : { text: params.text }), ...(params.ast === undefined ? {} : { ast: params.ast }), + ...(params.discover === undefined ? {} : { discover: params.discover }), language: params.language ?? (pattern ? "legacy" : "lucene"), signal: defaults.signal, ...(params.timezone === undefined ? {} : { timezone: params.timezone }), @@ -175,6 +184,7 @@ export const answerMessagesSearch = async ( ...threadArgs(args), ...(args.text === undefined ? {} : { text: args.text }), ...(args.ast === undefined ? {} : { ast: args.ast }), + ...(args.discover === undefined ? {} : { discover: args.discover }), language: args.language ?? "lucene", signal: defaults.signal, ...(args.timezone === undefined ? {} : { timezone: args.timezone }), diff --git a/src/search/question-plan.test.ts b/src/search/question-plan.test.ts new file mode 100644 index 000000000..1c1999448 --- /dev/null +++ b/src/search/question-plan.test.ts @@ -0,0 +1,67 @@ +import assert from "node:assert/strict" +import { test } from "vitest" +import { planQuestion } from "./question-plan.js" + +test("question scaffolding changes while content and filters survive", () => { + const plan = planQuestion("How many days are logs kept for Canyon log retention? chat:301 from:700") + assert.ok(plan.changed) + assert.equal(plan.scope, "chat:301 from:700") + assert.ok(plan.queries.every((q) => q.includes("days") && q.includes("canyon") && q.endsWith("chat:301 from:700"))) + assert.ok(plan.queries.some((q) => !q.includes("kept"))) +}) +test("unknown content is never removed or guessed from a vocabulary", () => { + const plan = planQuestion("How many ultraviolet logs are kept for Canyon log retention? chat:301") + assert.ok(plan.queries.every((q) => q.includes("ultraviolet"))) + const auditor = planQuestion("Who is the external auditor for Canyon refund amount? chat:301") + assert.ok(auditor.queries.every((q) => q.includes("external auditor"))) +}) +test("Russian question handling preserves numeric and scope requirements", () => { + const plan = planQuestion("Сколько дней хранятся логи по Опал хранение логов? chat:305 date:2026-10-08") + assert.ok(plan.changed) + assert.ok( + plan.queries.every((q) => q.includes("днеи") && q.includes("опал") && q.endsWith("chat:305 date:2026-10-08")), + ) +}) +test("explicit operators, phrases, wildcards and SDK strict flags stay unchanged", () => { + for (const q of [ + "Canyon AND retention", + '"Canyon log retention"', + "Canyon reten*", + "Canyon OR retention", + "What is Canyon AND retention?", + ]) + assert.deepEqual(planQuestion(q).queries, [q]) + assert.deepEqual(planQuestion("How many logs?", true).queries, ["How many logs?"]) + assert.deepEqual(planQuestion("", true).queries, [""]) +}) +test("an empty question cannot turn into a scope-only search", () => { + const raw = "What is? chat:301" + assert.deepEqual(planQuestion(raw).queries, [raw]) +}) + +test("Russian indirect questions retain permissions and unknown role qualifiers", () => { + const plan = planQuestion("Кому сейчас разрешён Опал доступ на сервер? chat:305") + assert.ok(plan.changed) + assert.ok(plan.queries.every((q) => !q.includes("кому") && q.includes("разрешен"))) + const unknown = planQuestion("Кому из внешних аудиторов разрешён доступ? chat:305") + assert.ok(unknown.queries.every((q) => q.includes("внешних") && q.includes("аудиторов"))) +}) + +test("temporal content is not discarded as generic question scaffolding", () => { + const plan = planQuestion("What happened after Canyon rollout? chat:301") + assert.ok(plan.queries.every((q) => q.includes("after"))) +}) + +test("ordinary questions do not require a question mark", () => { + assert.deepEqual( + planQuestion("Who currently has Canyon production access chat:301").queries, + planQuestion("Who currently has Canyon production access? chat:301").queries, + ) + assert.ok(planQuestion("К кому теперь обращаться по Опал контакт инцидента").changed) +}) + +test("bounded retention aliases preserve unrelated qualifiers", () => { + const plan = planQuestion("Какой срок хранения логов по Опал ultraviolet?") + assert.ok(plan.queries.some((q) => q.includes("хранение") && !q.includes("срок"))) + assert.ok(plan.queries.every((q) => q.includes("ultraviolet"))) +}) diff --git a/src/search/question-plan.ts b/src/search/question-plan.ts new file mode 100644 index 000000000..19122f9ce --- /dev/null +++ b/src/search/question-plan.ts @@ -0,0 +1,106 @@ +import { normalize } from "../store/normalize.js" +import { implicitTextTerms, parseLucene } from "./lucene/parser.js" +import { walkQuery } from "./lucene/types.js" + +export const words = (text: string) => normalize(text).match(/[\p{L}\p{N}]+/gu) ?? [] +const scaffolding = new Set( + words( + "what which how many much who where when is are was were do does did can could should we the a an for of to under from in on at has have сколько какой какая какие какое кто кому кем где когда как что во по на в к с у через ли должен должна нужно идёт идет", + ), +) +const questionWord = + /\b(?:what|which|how|who|where|when)\b|(?:^|\s)(?:сколько|какой|какая|какие|какое|кто|кому|кем|где|когда|как|что)(?:\s|$)/iu +export function planQuestion(text: string, strict = false) { + if (strict) + return { + changed: false, + natural: false, + queries: [text], + scope: "", + terms: words(text), + rankingTerms: words(text), + removed: [] as string[], + aliases: [] as string[], + } + const prefix = words(text).slice(0, 2) + const questionPrefix = + [ + "what", + "which", + "how", + "who", + "where", + "when", + "сколько", + "какои", + "какая", + "какие", + "какое", + "кто", + "кому", + "кем", + "где", + "когда", + "как", + "что", + ].includes(prefix[0] ?? "") || + (["к", "во"].includes(prefix[0] ?? "") && ["кому", "сколько"].includes(prefix[1] ?? "")) + const natural = questionWord.test(text) && (/\?/u.test(text) || questionPrefix) + const cleaned = natural ? text.replace(/\?(?=\s*(?:(?:chat|from|date):[^\s]+\s*)*$)/u, "") : text + const ast = parseLucene(cleaned) + const predicates = walkQuery(ast.root) + const textual = predicates.filter((p) => p.field === "text" || p.field === "exact") + const fields = predicates.filter((p) => p.field !== "text" && p.field !== "exact") + const scope = fields.map((p) => `${p.field}:${cleaned.slice(p.span.start, p.span.end)}`).join(" ") + const bypass = + strict || + !natural || + implicitTextTerms(cleaned).size === 0 || + textual.some((p) => p.operator !== "term" || p.field === "exact") + if (bypass) + return { + changed: false, + natural: false, + queries: [text], + scope, + terms: textual.flatMap((p) => words(p.value)), + rankingTerms: textual.flatMap((p) => words(p.value)), + removed: [] as string[], + aliases: [] as string[], + } + const original = textual.flatMap((p) => words(p.value)) + const removed = original.filter((w) => scaffolding.has(w)) + const terms = [...new Set(original.filter((w) => !scaffolding.has(w)))] + if (!terms.length) + return { + changed: false, + natural: true, + queries: [text], + scope, + terms, + rankingTerms: original, + removed, + aliases: [] as string[], + } + const alias = [...terms] + const changes: string[] = [] + const replace = (from: string[], to: string) => { + for (let i = 0; i < alias.length; i++) + if (from.includes(alias[i] ?? "")) { + changes.push(`${alias[i]} -> ${to}`) + alias[i] = to + } + } + if (terms.some((w) => ["refund", "euros", "euro", "money", "возврата", "возврат", "евро"].includes(w))) { + replace(["returned", "refunded", "received"], "refund") + replace(["вернули", "возвращено", "перечислено"], "возврат") + } + if (terms.some((w) => ["log", "logs", "retention", "backup", "логов", "логи", "хранение", "хранения"].includes(w))) { + replace(["lifetime", "kept", "keep"], "retention") + replace(["хранятся", "хранят", "хранить", "срок"], "хранение") + } + if (terms.includes("service") || terms.includes("build")) replace(["currently", "now"], "current") + const render = (ws: string[]) => [...new Set(ws)].join(" ") + (scope ? ` ${scope}` : "") + const queries = [...new Set([render(terms), render(alias)])].slice(0, 2) + return { changed: true, natural: true, queries, scope, terms, rankingTerms: original, removed, aliases: changes } +} diff --git a/src/services/messages-combined.ts b/src/services/messages-combined.ts index cef7eb0f2..85cb551bd 100644 --- a/src/services/messages-combined.ts +++ b/src/services/messages-combined.ts @@ -1,4 +1,4 @@ -import { CliError } from "@leemour/cli-core" +import { CliError } from "@wirecat/cli-core" import type { Messenger } from "../cli/messenger/context.js" import { parseLocator } from "../domain/locator.js" import type { Page } from "../domain/models.js" diff --git a/src/services/messages-discovery.test.ts b/src/services/messages-discovery.test.ts new file mode 100644 index 000000000..156693491 --- /dev/null +++ b/src/services/messages-discovery.test.ts @@ -0,0 +1,198 @@ +import { mkdtempSync } from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { afterEach, describe, expect, it, vi } from "vitest" +import type { Message } from "../domain/models.js" +import { parseLucene } from "../search/lucene/parser.js" +import { QUERY_LIMITS } from "../search/lucene/types.js" +import type { StoreContext } from "../store/sqlite/open.js" +import { directReplies } from "../store/sqlite/search.js" +import { type MessageStore, openStore } from "../store/store.js" +import { searchStore } from "./messages.js" + +const account = { provider: "synthetic", account: "500" } +const stores: MessageStore[] = [] +afterEach(async () => { + for (const s of stores.splice(0)) await s.close() +}) +const msg = (id: string, text: string, extra: Partial = {}): Message => ({ + id, + text, + chatId: "101", + senderId: "700", + senderName: "Synthetic", + timestamp: "2026-10-08T12:00:00.000Z", + editedAt: null, + outgoing: false, + attachments: [], + replyTo: null, + forwardedFrom: null, + reactions: null, + ...extra, +}) +const open = async () => { + const store = await openStore({ path: join(mkdtempSync(join(tmpdir(), "discovery-test-")), "synthetic.db") }) + stores.push(store) + await store.saveChats( + account, + ["101", "102"].map((id) => ({ + id, + title: "Synthetic", + kind: "group" as const, + unreadCount: 0, + lastMessageAt: null, + participantsCount: 10, + })), + ) + await store.saveMessages( + account, + "101", + [ + msg("1", "Helix export: what time should the daily export run?"), + msg("2", "At 06:45 UTC.", { replyToId: "1" }), + msg("3", "At 02:00 UTC.", { replyToId: "1", senderId: "701" }), + msg("4", "At 01:00 UTC.", { replyToId: "1", timestamp: "2026-10-07T12:00:00.000Z" }), + msg("5", "Helix logs expire after 21 days."), + ], + { via: "synthetic" }, + ) + await store.fillSearchIndex({}) + await store.fillStems({}) + return store +} +const request = (text: string) => ({ text, discover: true, language: "lucene" as const, limit: 10, timezone: "UTC" }) +const ids = (result: { items: { id: string }[] }) => result.items.map((m) => m.id) + +describe("archive discovery", () => { + it("finds partial wording and direct replies without a conversation rebuild or archive snapshot", async () => { + const store = await open() + const found = await searchStore(store, account, request("What time does Helix export run?")) + expect(ids(found)).toContain("2") + expect(found.items.find((m) => m.id === "2")?.discovery?.parent).toBe("msg:synthetic/500/101/1") + expect(found.query?.discovery).toMatchObject({ method: "lexical-partial", repliesAdded: 3 }) + const partial = await searchStore(store, account, request("What is Helix ultraviolet retention?")) + expect(partial.items.length).toBeGreaterThan(0) + expect(partial.items.every((m) => m.discovery?.missingTerms.includes("ultraviolet"))).toBe(true) + expect( + (await searchStore(store, account, { ...request("What time does Helix export run?"), discover: false })).items, + ).toEqual([]) + }) + it("preserves used correction provenance for keyword discovery", async () => { + const store = await open() + await store.saveMessages(account, "101", [msg("6", "Helix rollout confirmed")], { via: "synthetic" }) + await store.fillSearchIndex({}) + await store.fillStems({}) + const found = await searchStore(store, account, request("Helix rolluot")) + expect(found.items.find((m) => m.id === "6")?.match).toBe("corrected") + expect(found.corrections).toEqual([{ from: "rolluot", to: ["rollout"] }]) + }) + it("retains eligible replies when the lexical shortlist is full", async () => { + const store = await open() + await store.saveMessages( + account, + "101", + Array.from({ length: 400 }, (_, n) => msg(String(2000 + n), "Helix export discussion heading")), + { via: "synthetic" }, + ) + await store.fillSearchIndex({}) + await store.fillStems({}) + const found = await searchStore(store, account, request("What time does Helix export run?")) + expect(ids(found)).toContain("2") + expect(found.query?.discovery?.candidates).toBeLessThanOrEqual(300) + expect(found.query?.discovery?.truncated).toBe(true) + }) + it("checks both parent and reply against sender, date, chat and only filters", async () => { + const store = await open() + const scoped = await searchStore( + store, + account, + request("What time does Helix export run? chat:101 from:700 date:2026-10-08"), + ) + expect(ids(scoped)).toContain("2") + expect(ids(scoped)).not.toContain("3") + expect(ids(scoped)).not.toContain("4") + const childOnly = await searchStore(store, account, { + ...request("What time does Helix export run?"), + only: [{ chatId: "101", id: "2" }], + }) + expect(childOnly.items).toEqual([]) + const parentOnly = await searchStore(store, account, { + ...request("What time does Helix export run?"), + only: [{ chatId: "101", id: "1" }], + }) + expect(ids(parentOnly)).toEqual(["1"]) + const fromChild = await searchStore(store, account, request("What time does Helix export run? from:701")) + expect(ids(fromChild)).not.toContain("3") + }) + it("isolates accounts/chats and excludes deleted parents and children", async () => { + const store = await open() + const other = { provider: "synthetic", account: "600" } + await store.saveChats(other, [ + { id: "101", title: "Other", kind: "group", unreadCount: 0, lastMessageAt: null, participantsCount: 10 }, + ]) + await store.saveMessages( + other, + "101", + [msg("1", "Helix export foreign parent"), msg("9", "Foreign reply", { replyToId: "1" })], + { via: "synthetic" }, + ) + await store.saveMessages( + account, + "102", + [ + msg("1", "Other project export", { chatId: "102" }), + msg("9", "Other chat reply", { chatId: "102", replyToId: "1" }), + ], + { via: "synthetic" }, + ) + await store.markDeleted(account, ["3"], { chatId: "101" }) + const replies = await store.directReplies?.([{ account, chatId: "101", id: "1" }], 100) + expect(replies?.items.map((m) => m.id)).toEqual(["2", "4"]) + await store.markDeleted(account, ["1"], { chatId: "101" }) + expect((await store.directReplies?.([{ account, chatId: "101", id: "1" }], 100))?.items).toEqual([]) + }) + it.each([ + "Helix AND export", + '"Helix export"', + "Helix OR logs", + "Helix NOT logs", + "exact:Helix", + "Helix*", + "(Helix export)", + ])("preserves explicit syntax %s", async (text) => { + const store = await open() + const strict = await searchStore(store, account, { ...request(text), discover: false }) + expect(await searchStore(store, account, request(text))).toEqual(strict) + }) + it("preserves exact/newest/AST and surrounding context", async () => { + const store = await open() + for (const strict of [{ exact: true }, { newest: true }, { text: undefined, ast: parseLucene("Helix export") }]) { + const q = { ...request("Helix export"), ...strict } + expect(await searchStore(store, account, q)).toEqual(await searchStore(store, account, { ...q, discover: false })) + } + expect( + (await searchStore(store, account, { ...request("What time does Helix export run?"), context: 1 })).items[0] + ?.context?.length, + ).toBeGreaterThan(0) + }) + it("enforces input, cancellation and bounded reply bodies", async () => { + const store = await open() + for (const extra of [ + { limit: 0 }, + { context: 21 }, + { backend: "server" as const }, + { language: "legacy" as const }, + { pattern: /export/ }, + ]) + await expect(searchStore(store, account, { ...request("Helix export"), ...extra })).rejects.toThrow() + const controller = new AbortController() + controller.abort() + await expect( + searchStore(store, account, { ...request("Helix export"), signal: controller.signal }), + ).rejects.toThrow("aborted") + const context = { + database: { prepare: vi.fn(() => ({ all: () => [{ pk: 1, bytes: QUERY_LIMITS.bodyBytes + 1 }] })) }, + } as unknown as StoreContext + expect(() => directReplies(context, [{ account, chatId: "101", id: "1" }], 100)).toThrow("bodyBytes") + }) +}) diff --git a/src/services/messages-discovery.ts b/src/services/messages-discovery.ts new file mode 100644 index 000000000..66c9c50d7 --- /dev/null +++ b/src/services/messages-discovery.ts @@ -0,0 +1,190 @@ +import { CliError } from "@wirecat/cli-core" +import type { Messenger } from "../cli/messenger/context.js" +import { parseLocator } from "../domain/locator.js" +import { implicitTextTerms } from "../search/lucene/parser.js" +import { exhausted, QUERY_LIMITS, walkQuery } from "../search/lucene/types.js" +import { planQuestion, words } from "../search/question-plan.js" +import type { AccountKey, MessageStore } from "../store/store.js" +import type { FoundMessage, SearchFound, SearchQuery } from "./messages.js" +import { searchCombined } from "./messages-combined.js" +import { prepareLucene, searchLucene } from "./messages-search.js" + +export const searchDiscovery = async ( + store: MessageStore, + account: AccountKey, + request: SearchQuery, + messenger: Partial> = {}, +): Promise => { + const original = await prepareLucene(store, account, request, messenger) + const text = request.text ?? "" + const plan = planQuestion(text, request.ast !== undefined || !!request.exact || !!request.newest) + if (request.ast !== undefined || request.exact || request.newest || implicitTextTerms(text).size === 0) + return searchLucene(store, account, request, messenger) + const implicit = implicitTextTerms(text) + if ( + !plan.changed && + walkQuery(original.execution.root).some( + (node) => + (node.field === "text" || node.field === "exact") && + (node.operator !== "term" || node.field === "exact" || !implicit.has(node.span.start)), + ) + ) + return searchLucene(store, account, request, messenger) + const terms = [...new Set(plan.terms.concat(plan.aliases.flatMap((alias) => words(alias.split(" -> ")[1] ?? ""))))] + if (!terms.length) return searchLucene(store, account, request, messenger) + if (terms.length > 64) exhausted("discovery terms") + const started = performance.now() + const check = () => { + if (request.signal?.aborted) + throw new CliError("validation_error", "search was aborted", { reason: "query_aborted", complete: false }) + if (performance.now() - started > QUERY_LIMITS.milliseconds) exhausted("milliseconds") + } + const union = new Map() + let truncated = false + const corrections = new Map() + const add = (items: FoundMessage[]) => + items.forEach((m, rank) => { + const previous = union.get(m.locator) + union.set(m.locator, { ...previous, ...m, fused: (previous?.fused ?? 0) + 1 / (60 + rank + 1) }) + }) + // Natural questions use one ranked partial lookup; repeated all-word lookups cannot rescue their wording gaps. + for (const query of plan.changed ? [] : plan.queries) { + check() + const found = await searchCombined(store, account, { ...request, text: query, limit: 300 }, messenger, { + candidateDepth: 300, + rrfK: 60, + rerank: "none", + chatCap: 0, + }) + add(found.items) + for (const correction of found.corrections) corrections.set(correction.from, correction) + truncated ||= found.hasMore || found.query.combined.truncated + } + check() + const relaxed = `(${terms.join(" OR ")})${plan.scope ? ` ${plan.scope}` : ""}` + const partialRequest = { ...request, text: relaxed, ast: undefined, limit: 300, context: 0, thread: undefined } + const partial = await searchLucene(store, account, partialRequest, messenger) + add(partial.items) + truncated ||= partial.hasMore || union.size > 300 + let candidates = [...union.values()].sort((a, b) => b.fused - a.fused).slice(0, 300) + const originals = new Set(candidates.map((m) => m.locator)) + const parentByLocator = new Map() + let replyCount = 0 + if (store.directReplies && candidates.length) { + check() + const proposed = await store.directReplies( + candidates.map((m) => { + const locator = parseLocator(m.locator) + return { account: { provider: locator.provider, account: locator.account }, chatId: m.chatId, id: m.id } + }), + 100, + ) + truncated ||= proposed.hasMore + if (proposed.items.length) { + const only = candidates + .concat(proposed.items.map((m) => ({ ...m, fused: 0 }))) + .filter((m) => !request.only || request.only.some((key) => key.chatId === m.chatId && key.id === m.id)) + .map((m) => ({ chatId: m.chatId, id: m.id })) + const eligible = await searchLucene( + store, + account, + { + ...request, + text: plan.scope || undefined, + ast: undefined, + limit: 500, + only, + context: 0, + thread: undefined, + }, + messenger, + ) + if (eligible.hasMore) exhausted("reply eligibility") + const allowed = new Set(eligible.items.map((m) => m.locator)) + const parents = new Map( + candidates.map((m) => [ + JSON.stringify([m.chatId, m.id, parseLocator(m.locator).account, parseLocator(m.locator).provider]), + m, + ]), + ) + for (const child of proposed.items) { + check() + const owner = parseLocator(child.locator) + const parent = parents.get(JSON.stringify([child.chatId, child.replyToId, owner.account, owner.provider])) + if (!parent || !allowed.has(parent.locator) || !allowed.has(child.locator)) continue + parentByLocator.set(child.locator, parent) + if (!candidates.some((m) => m.locator === child.locator)) { + candidates.push({ ...child, fused: parent.fused }) + replyCount++ + } + } + } + } + truncated ||= candidates.length > 300 + candidates = candidates.sort((a, b) => b.fused - a.fused).slice(0, 300) + replyCount = candidates.filter((m) => !originals.has(m.locator)).length + const stem = (w: string) => original.execution.stemmer?.stemToken(w) ?? w + const stems = terms.map(stem) + let bytes = 0 + candidates = candidates + .map((m) => { + check() + const parent = parentByLocator.get(m.locator) + const body = `${parent ? `${parent.text}\n` : ""}${m.text}` + bytes += Buffer.byteLength(body) + if (bytes > QUERY_LIMITS.bodyBytes) exhausted("bodyBytes") + const tokens = new Set(words(body).map(stem)) + const matchedTerms = terms.filter((_, i) => tokens.has(stems[i] ?? "")) + const missingTerms = terms.filter((_, i) => !tokens.has(stems[i] ?? "")) + const coverage = matchedTerms.length / terms.length + return { + ...m, + score: coverage * 4 + m.fused, + discovery: { + coverage, + matchedTerms, + missingTerms, + ...(parent ? { parent: parent.locator } : {}), + }, + } + }) + .sort( + (a, b) => + (b.score ?? 0) - (a.score ?? 0) || + Number(parentByLocator.has(b.locator)) - Number(parentByLocator.has(a.locator)), + ) + check() + const items = await Promise.all( + candidates.slice(0, request.limit).map(async ({ fused: _fused, ...m }) => { + if (!request.context) return m + check() + const owner = parseLocator(m.locator) + return { + ...m, + context: await store.around({ provider: owner.provider, account: owner.account }, m.chatId, m.id, { + before: request.context, + after: request.context, + }), + } + }), + ) + if (!partial.query) throw new Error("Lucene discovery query metadata is missing") + return { + ...partial, + corrections: items.some((m) => m.match === "corrected") ? [...corrections.values()] : [], + items, + hasMore: truncated || candidates.length > request.limit, + query: { + ...partial.query, + discovery: { + method: "lexical-partial", + candidateDepth: 300, + candidates: candidates.length, + repliesAdded: replyCount, + truncated, + terms, + queries: [...plan.queries, relaxed], + }, + }, + } +} diff --git a/src/services/messages-search.ts b/src/services/messages-search.ts index bffaa9c6f..1fe34074d 100644 --- a/src/services/messages-search.ts +++ b/src/services/messages-search.ts @@ -23,6 +23,15 @@ import type { SearchRefreshed } from "./search-refresh.js" export interface QueryMetadata { language: "lucene-v1" + discovery?: { + method: "lexical-partial" + candidateDepth: number + candidates: number + repliesAdded: number + truncated: boolean + terms: string[] + queries: string[] + } version: 1 fieldsVersion: number presetVersion: number diff --git a/src/services/messages.ts b/src/services/messages.ts index 80e091166..ba5a4248d 100644 --- a/src/services/messages.ts +++ b/src/services/messages.ts @@ -25,6 +25,7 @@ import type { WordQuery, } from "../store/store.js" import { fromStore, nothingStored, PUSHED, type ServiceDeps } from "./deps.js" +import { searchDiscovery } from "./messages-discovery.js" import { type MessageStats, type QueryMetadata, @@ -74,6 +75,7 @@ export interface SearchQuery { only?: { chatId: Id; id: Id }[] /** Strict Lucene by default; legacy discovery is explicit through language or a RegExp pattern. */ text?: string + discover?: boolean language?: "lucene" | "legacy" timezone?: string ast?: unknown @@ -100,6 +102,7 @@ export interface SearchQuery { } export type FoundMessage = StoredHit & { + discovery?: { coverage: number; matchedTerms: string[]; missingTerms: string[]; parent?: string } match?: Match score?: number | null exact?: boolean @@ -421,11 +424,13 @@ export const messagesService = (deps: ServiceDeps): MessagesService => { }), search: (request) => { + if (request.discover && request.backend === "server") + throw new CliError("validation_error", "discovery searches the local archive; use backend archive") const query: SearchQuery = { ...request, language: request.language ?? (request.pattern ? "legacy" : "lucene"), // Mail is only ever in the local store, and a saved search replays as `search messages`. - ...(request.kind === "mail" ? { backend: "archive" as const } : {}), + ...(request.kind === "mail" || request.discover ? { backend: "archive" as const } : {}), } return inStore(async (store, account) => { if (query.thread) threadBounds(query.thread) @@ -720,7 +725,14 @@ export const searchStore = async ( throw new CliError("validation_error", "search was aborted", { reason: "query_aborted", complete: false }) validateSearchDialect(request) if (request.thread) threadBounds(request.thread) + if (request.discover !== undefined && typeof request.discover !== "boolean") + throw new CliError("validation_error", "discover takes a boolean") + if (request.discover && (pattern || request.language === "legacy")) + throw new CliError("validation_error", "discovery requires Lucene text, without legacy or regex") + if (request.discover && request.backend === "server") + throw new CliError("validation_error", "discovery searches the local archive; use backend archive") await topUp(store) + if (request.discover) return withThreads(store, await searchDiscovery(store, account, request, messenger), request) if (!pattern && (request.language === "lucene" || request.ast !== undefined)) return withThreads(store, await searchLucene(store, account, request, messenger), request) const found = pattern @@ -1004,6 +1016,7 @@ const paramsOf = (command: SearchCommand, query: SearchQuery & { by?: StatsGroup ? {} : { text: query.text }), ...(query.ast === undefined ? {} : { ast: query.ast }), + ...(query.discover === undefined ? {} : { discover: query.discover }), ...(query.language === undefined ? {} : { language: query.language }), ...(query.chat === undefined ? {} : { chat: query.chat }), ...(query.source === undefined ? {} : { source: query.source }), diff --git a/src/services/searches.ts b/src/services/searches.ts index e6e6bf965..5e32bbd24 100644 --- a/src/services/searches.ts +++ b/src/services/searches.ts @@ -11,6 +11,7 @@ import type { StatsGrouping } from "./messages-search.js" /** What a run or a saved search keeps: the query and options as given, never a message or a result. */ export interface SearchParams extends RankingInput { + discover?: boolean adminReport?: AdminReport selection?: unknown target?: RankingTarget @@ -100,6 +101,8 @@ const patternOf = (source: string): RegExp => { /** The query checked as a search would check it, without the store: syntax, fields, values. */ const checked = (params: SearchParams): SearchParams => { + if (params.discover && (params.regex || languageOf(params) === "legacy")) + throw new CliError("validation_error", "discovery requires Lucene text, without legacy or regex") if (params.text !== undefined && params.ast !== undefined) throw new CliError("validation_error", "give query text or an AST, not both") if (params.regex) { diff --git a/src/store/sqlite/lucene.ts b/src/store/sqlite/lucene.ts index a1cabaaca..e7ebfbfd8 100644 --- a/src/store/sqlite/lucene.ts +++ b/src/store/sqlite/lucene.ts @@ -412,8 +412,11 @@ const compileQuery = (context: StoreContext, execution: QueryExecution, boundedA [ where, { - sql: "EXISTS (SELECT 1 FROM json_each(?) j WHERE json_extract(j.value,'$.chatId')=c.native_id AND json_extract(j.value,'$.id')=m.native_id)", - params: [JSON.stringify(execution.only)], + sql: `m.pk IN (SELECT picked.pk FROM json_each(?) j CROSS JOIN accounts allowed CROSS JOIN chats scoped CROSS JOIN messages picked + WHERE (${execution.accounts.map(() => "(allowed.provider=? AND allowed.native_id=?)").join(" OR ") || "0"}) + AND scoped.account_pk=allowed.pk AND scoped.native_id=json_extract(j.value,'$.chatId') + AND picked.chat_pk=scoped.pk AND picked.native_id=json_extract(j.value,'$.id'))`, + params: [JSON.stringify(execution.only), ...execution.accounts.flatMap((key) => [key.provider, key.account])], exact: true, }, ], diff --git a/src/store/sqlite/manifest.ts b/src/store/sqlite/manifest.ts index ba1176258..e95a5e84a 100644 --- a/src/store/sqlite/manifest.ts +++ b/src/store/sqlite/manifest.ts @@ -52,6 +52,7 @@ export const MANIFEST: ManifestEntry[] = [ { name: "20261008191711_version-27-owner-targets", version: 27, minCompatible: 6 }, { name: "20261009092409_version-28-legacy-notes-data", version: 28, minCompatible: 28 }, { name: "20261009092436_version-28-drop-legacy-notes", version: 28, minCompatible: 28 }, + { name: "20261010132610_version-29-direct-replies", version: 29, minCompatible: 28 }, ] export const generatedMigrations = ( diff --git a/src/store/sqlite/migrations.generated.ts b/src/store/sqlite/migrations.generated.ts index 895d42ea1..ec4d10820 100644 --- a/src/store/sqlite/migrations.generated.ts +++ b/src/store/sqlite/migrations.generated.ts @@ -365,5 +365,11 @@ export const GENERATED: { name: string; statements: string[] }[] = [ "DROP TABLE `knowledge_relations`;", "DROP TABLE `knowledge_targets`;" ] + }, + { + "name": "20261010132610_version-29-direct-replies", + "statements": [ + "CREATE INDEX `messages_by_reply` ON `messages` (`chat_pk`,`reply_to_native_id`) WHERE reply_to_native_id IS NOT NULL AND deleted_at IS NULL;" + ] } ] diff --git a/src/store/sqlite/schema.ts b/src/store/sqlite/schema.ts index bbee9e328..6bbe1d807 100644 --- a/src/store/sqlite/schema.ts +++ b/src/store/sqlite/schema.ts @@ -156,6 +156,9 @@ export const messages = sqliteTable( index("messages_by_time").on(table.chatPk, desc(table.sentAt)), index("messages_by_account").on(table.accountPk, table.nativeId), index("messages_by_sender").on(table.senderIdentityPk), + index("messages_by_reply") + .on(table.chatPk, table.replyToNativeId) + .where(sql`reply_to_native_id IS NOT NULL AND deleted_at IS NULL`), // Empty once the backfill is done, so every open can ask "anything left?" without reading the table. index("messages_to_normalize").on(table.pk).where(sql`normalized_text IS NULL AND deleted_at IS NULL`), ], diff --git a/src/store/sqlite/search.ts b/src/store/sqlite/search.ts index 245545ff0..10d5ba2dc 100644 --- a/src/store/sqlite/search.ts +++ b/src/store/sqlite/search.ts @@ -1,7 +1,8 @@ import { CliError } from "@wirecat/cli-core" import { formatLocator } from "../../domain/locator.js" import type { Message, Page } from "../../domain/models.js" -import type { MessageFilter, StoredHit } from "../store.js" +import { exhausted, QUERY_LIMITS } from "../../search/lucene/types.js" +import type { AccountKey, MessageFilter, StoredHit } from "../store.js" import { alias, and, desc, eq, inArray, isNull, lte, type SQL, sql } from "./drizzle/core.js" import type { StoreContext } from "./open.js" import { MESSAGE_FIELDS, type MessageRow, newestFirst, toMessages } from "./reads.js" @@ -169,3 +170,34 @@ const perChatNewest = (context: StoreContext, where: SQL | undefined, wanted: nu .orderBy(desc(ranked.sentAt), desc(ranked.pk)) .all() } + +export const directReplies = ( + context: StoreContext, + parents: { account: AccountKey; chatId: string; id: string }[], + limit: number, +): Page => { + if (!Number.isInteger(limit) || limit < 1 || limit > 100 || parents.length > 300) + throw new CliError("validation_error", "direct reply lookup needs 1–100 results and at most 300 parents") + if (!parents.length) return { items: [], hasMore: false } + const rows = context.database + .prepare(` + SELECT DISTINCT m.pk, length(cast(m.text AS BLOB)) AS bytes FROM json_each(?) requested + JOIN accounts ac ON ac.provider=json_extract(requested.value,'$.account.provider') + AND ac.native_id=json_extract(requested.value,'$.account.account') + JOIN chats c ON c.account_pk=ac.pk AND c.native_id=json_extract(requested.value,'$.chatId') + JOIN messages parent ON parent.chat_pk=c.pk AND parent.native_id=json_extract(requested.value,'$.id') + JOIN messages m INDEXED BY messages_by_reply ON m.chat_pk=c.pk AND m.reply_to_native_id=parent.native_id + WHERE m.deleted_at IS NULL AND parent.deleted_at IS NULL AND m.reply_to_native_id IS NOT NULL + ORDER BY cast(requested.key AS INTEGER), m.pk LIMIT ? + `) + .all(JSON.stringify(parents), limit + 1) + if (rows.slice(0, limit).reduce((sum, row) => sum + Number(row.bytes), 0) > QUERY_LIMITS.bodyBytes) + exhausted("bodyBytes") + return { + items: hitsByPk( + context, + rows.slice(0, limit).map((r) => Number(r.pk)), + ), + hasMore: rows.length > limit, + } +} diff --git a/src/store/store.ts b/src/store/store.ts index 3e49f7407..fe1d25b95 100644 --- a/src/store/store.ts +++ b/src/store/store.ts @@ -417,6 +417,7 @@ export interface MessageStore { ): Promise adminStatisticsQuery?(execution: QueryExecution, request: AdminStoreRequest): Promise rankQuery?(execution: QueryExecution, request: RankingRequest): Promise + directReplies?(parents: { account: AccountKey; chatId: Id; id: Id }[], limit: number): Promise> matchQuery?(execution: QueryExecution): Promise> conversationEligibility?(execution: QueryExecution): Promise /** The same matches as `matchQuery`, each counted once, grouped by chat, sender or quarter hour. */ @@ -1286,6 +1287,7 @@ const storeOver = (context: StoreContext): MessageStore => { }, adminStatisticsQuery: async (execution, request) => adminStatisticsQuery(context, execution, request), rankQuery: async (execution, request) => rankQuery(context, execution, request), + directReplies: async (parents, limit) => search.directReplies(context, parents, limit), matchQuery: async (execution) => lucene.matchQuery(context, execution), conversationEligibility: async (execution) => conversationEligibility(context, execution), countQuery: async (execution, by) => lucene.countQuery(context, execution, by), From d73d190b19bba9d25ef746aeb13f49a03cb5d590 Mon Sep 17 00:00:00 2001 From: leemour Date: Sat, 10 Oct 2026 15:49:31 +0200 Subject: [PATCH 10/14] docs(search): register archive discovery in the parity catalogue --- docs/dev/STANDARD.md | 1 + parity.json | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/docs/dev/STANDARD.md b/docs/dev/STANDARD.md index 0ea2b2b17..302b984bc 100644 --- a/docs/dev/STANDARD.md +++ b/docs/dev/STANDARD.md @@ -374,6 +374,7 @@ its meaning, and this table is regenerated from it. **Bold** marks a clash still | `--deny-people` | `` | replace denied sender ids, comma-separated; empty clears; deny wins | | `replies audience` | | `--description` | `` | the new about text — of a chat or of your account | | `account update`, `chats update` | | `--dims` | `` | remote: the vector size — needed with --base-url; shortens an OpenAI model's | | `conversations embed`, `conversations embed clear`, `conversations embed status`, `conversations related`, `conversations status`, `search conversations` | +| `--discover` | | find partial lexical matches and eligible replies in the local archive; results are evidence, not confirmed answers | | `search messages` | | `--do` | `` | actions: reply, task, or both, comma-separated | | `replies edit` | | `--download` | | first save, from the messenger, the files no download saved yet | | `attachments extract` | | `--dry-run` | | judge and plan; do nothing. **A command-specific dry run keeps its richer plan; otherwise the shared shell previews arguments and permissions without running the action. Targets remain unresolved unless that command provides a preview.** | | every command, `bot chats moderate`, `chats moderate` (planned), `config migrate`, `stats messages counters refresh`, `store repair`, `tags auto` | diff --git a/parity.json b/parity.json index 3a074f31c..b402339dc 100644 --- a/parity.json +++ b/parity.json @@ -293,6 +293,9 @@ "value": "", "meaning": "remote: the vector size — needed with --base-url; shortens an OpenAI model's" }, + "--discover": { + "meaning": "find partial lexical matches and eligible replies in the local archive; results are evidence, not confirmed answers" + }, "--do": { "meaning": "actions: reply, task, or both, comma-separated", "value": "" @@ -3366,6 +3369,7 @@ }, "--chat": "all", "--context": "all", + "--discover": "all", "--exact": "all", "--language": "all", "--limit": "all", From a84510645f0ea1254d2c543640b1290a3f0b1a0f Mon Sep 17 00:00:00 2001 From: leemour Date: Sat, 10 Oct 2026 15:52:10 +0200 Subject: [PATCH 11/14] docs(search): explain discovery archive override in command help --- docs/dev/STANDARD.md | 2 +- parity.json | 2 +- src/cli/messenger/messages-search-command.ts | 4 ++-- src/cli/messenger/search-backend-options.ts | 2 +- src/mcp/tools/search.ts | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/dev/STANDARD.md b/docs/dev/STANDARD.md index 302b984bc..7f100c50c 100644 --- a/docs/dev/STANDARD.md +++ b/docs/dev/STANDARD.md @@ -328,7 +328,7 @@ its meaning, and this table is regenerated from it. **Bold** marks a clash still | `--as-reply` | | send as a reply to the matched message | | `replies edit` | | `--at-time` | `