[lyesrock]

Problem

On a machine whose drafter the auto rule turns off, decode is one token a round even when the model is copying: repeating a list, rewriting a file, echoing a document. DeepSeek-V4.1-Flash with the IST-DAS 2.25 bpw pack on an RTX 5090 is exactly that case - the engine itself says so (DSpark not loaded: the routed experts (149 GB) are far more than the GPU holds (33 GB)), and its continuation lives in the request, past any drafter's block. Measured there: reproducing 48 commands, 620 tokens, 619 rounds, decode 39.8 tok/s.

Result (opt-in, default path unchanged)

DS4_LOOKUP=1 drafts each round from the request's own history. Same reproduction: 39.8 -> 68-70 tok/s (+74%), 35-46 rounds instead of 619, and the 300-token check is byte-identical off/on. Real agent traffic (a 220 000-token coding conversation at temperature 1.0) went 37-39 -> 43 tok/s with 2041 tokens accepted from the lookup; short prompts and warm prose are unchanged within run-to-run spread.

Approach

Lookup keeps an incrementally maintained bigram chain over everything the request has read and written (a 220 000-token conversation costs microseconds a round). Each round its tail's last two tokens find their earlier occurrences, the copy is extended back up to 64 tokens, and what followed the match is proposed as the round's block. The proposal goes through the trunk's own verify, greedy against its argmax and sampled through the same point-mass rejection the drafter's argmax drafts already use - the model keeps every token, nothing here decides one. With the drafter loaded the two coexist (a long match wins the block, or the sources agree on the first token). On a drafter-less machine the window rollback buffers and the verify scratch are allocated for lookup too (small, and only when the knob is set), and an empty round no longer cools the request (a plain step always offers nothing). A guard keeps prose out of the window: the match must be 12 tokens or more (16 after two empty rounds; 4 once a copy runs). DS4_LK_STATS=1 prints rounds, kept drafts and the longest match.

Validation - RTX 5090 32 GB, 121 GB DDR5, Ryzen 9950X3D, engine 0.1.52 with this branch, pack IST-DAS 2.25 bpw, --mmap-experts off --prefill-chunk 8192 --dspark auto (auto off), fresh prompt-cache key per request, drop_caches between boots

Passed

• CUDA build of the submitted revision (commit 3aacc78).

• Lossless, submitted revision: stock 0.1.52 binary and this binary each booted --serve with the same args; the same GEN 300 greedy request (an 873-token prompt) answered with 300 T lines byte-identical (cmp).

• Reproduction, 48 commands, greedy, 620 tokens, cold boots: off 39.80 / 39.68 tok/s; on 68.32 / 69.37 / 70.17 tok/s; --spec-eval: 1.00 tok/round off vs 14.07 tok/round on.

• Same reproduction sampled at temperature 1.0: 85.2 tok/s, 981 of 3992 drafts kept.

• Real traffic (an agent at ~220 000 tokens): a 1232-token answer at 43.06 tok/s with 161 lookup rounds on the request, vs 37-39 tok/s for the same agent before the branch.

• Passed, hybrid with the drafter forced on (--dspark on - the only arm where the agree/override rule runs): same binary, DS4_LOOKUP=0 vs 1, one boot each: the 300-token greedy T lines byte-identical (cmp); the reproduction decodes 81.63 vs 94.07 tok/s cold (+15.2%) and 102.85 vs 112.34 warm (+9.2%), the lookup taking 58 rounds and 840 of 3430 proposals kept. The agreement check follows HyperQwen's _AGREE: the two sources must propose the same first two tokens.

Not run

• HIP and Metal builds: the touched files are the CUDA dsv41 engine's and the box has no ROCm - as in [Engine] DeepSeek prompt: the busy experts take FP16 GEMMs, not the tiles (DS4_FAT_ROWS, opt-in) #4366, CI's help asked.
• Byte-identity under sampling: point-mass rejection preserves the distribution, not a fixed sequence; the greedy path is what the cmp proves.

Known noise, for honesty: two stock 0.1.52 boots of the same request differ by one character at the max_tokens cutoff on their own (expert-tier copies drift per boot), so identity is asserted per boot-class, off vs on.

Limits

Off by default; one extra environment knob to learn. The 12-token floor is measured on the prose ramp and short prompts (unchanged), not exhaustively on all prose. The lookup can only help copies whose tail's bigram appeared before; anything new pays one proposal attempt.