[Mattral]

docs: engineering notes

Adds docs/lessons.md, titled "Engineering notes: making FlashSpec faster": the long engineering-notes version of the
FlashSpec write-up (11 lessons, 3,302 words, 9 figures). The draft's article-style title ("Making speculative decoding actually
faster: lessons from FlashSpec") is replaced, so the page reads as documentation, not a second copy of the post. The refined public article is separate and is not in this
repo. Do not merge until reviewed.

Files

• docs/lessons.md: the draft, unchanged except:
  • the 9 image paths are absolute URLs (https://raw.githubusercontent.com/Mattral/FlashSpec/main/docs/images/lessons/…).
    raw.githubusercontent.com serves SVG as image/svg+xml (checked), so they render on GitHub and in the docs build (built
    locally: site/lessons/index.html has all 9);
  • the sources line names scripts/article/ instead of "this folder", and says figures 2–3 have no generator;
  • one hardware sentence removed: "The development machine has no GPU." It isn't a model name, but it is a
    development-machine detail beyond "Tesla T4". The rest of that sentence ("Every GPU number comes from a Colab notebook…") is
    kept.
• docs/images/lessons/fig1…fig9.svg: copied unchanged.
• scripts/article/make_phase6_figures.py (figures 5–8) and scripts/article/make_030_figures.py (figures 4 and 9):
  • REPO = Path(__file__).resolve().parents[2] replaces the hard-coded home path;
  • output goes to docs/images/lessons/ by default (--out DIR to override);
  • figures() returns name → SVG text;
  • ruff clean: docstrings added, and two variables named l renamed. Drawing code and output are unchanged.
• Byte-for-byte:
  • run from /tmp into an empty directory, both scripts reproduce fig4–fig9 byte-identical to the committed SVGs;
  • tests/unit/test_article_figures.py now checks this on every CI run, and it fails if a single number in an SVG is edited
    (tried, then restored);
  • figures 1–3 have no generator script, so they are checked for presence only; their numbers are audited below.
• mkdocs.yml gets "Engineering notes: lessons.md" in the nav; llms.txt gets an entry under Docs.

Number audit

Every number in lessons.md and in the SVGs without a generator, against the source it cites. "ok" means the article's rounded
value equals the source value at the article's precision.

claim in lessons.md	source value	source	status
0.1.5: 0.61× plain decoding, T4	0.6061833262227975	tinyllama_gpu_flashspec_ucb_cb02ca1.json speedup_vs_ar_mean	ok
0.2.0: 1.08×–1.49× vs compiled HF	1.0837177249808096–1.4929247183661758	8d036d7 speedup_vs_*_compiled_median	ok
0.3.0: 1.26×–1.67× vs compiled HF	1.2630258753083248–1.6703915213699534	3c43e08	ok
Fig. 4: 2.6×–3.4× eager HF; 3.7×–4.5× HF assisted	2.5801491971418065–3.4316137147436416; 3.6799192622210697–4.474878921310762	3c43e08	ok
[8, 4, 1, 4, 1, …]	quoted verbatim from PLAN.md's toy forward-count audit	not a result JSON	ok (not from a JSON)
Fig. 1 break-even "around 0.7" → "about 0.4"	0.696 → 0.38 with the figure's stated γ=4, c=0.15	closed form	ok
c = 0.90 eager / 0.48 compiled; Fig. 3 31.7 / 35.4 ms, 9.1 / 19.1 ms	0.8978235042101232 / 0.47562151975963995; 31.739 / 35.351, 9.090 / 19.111 ms	8d036d7 forward_cost, MT-Bench T=0.7 cell	ok (the figure doesn't name the cell; the other three cells give c 0.884–0.904)
24 vs 28 layers, layer ratio 0.86	24 / 28 = 0.8571	Qwen2.5 config.json	ok
parameter ratio (0.33)	494,032,768 / 1,543,714,304 = 0.320	HF safetensors totals	mismatch (0.33 is the nominal 0.5B/1.5B; actual 0.32)
Fig. 2: eager 129 + 38 = 167 ms, drafting 77%; compiled 39 + 20 = 59 ms, "two-thirds"	128.55 + 38.03, 167.02 ms, 0.7697; 38.58 + 19.54, 58.58 ms, 0.6585	8d036d7 HumanEval greedy step_profile	ok
Fig. 8: lookup step 21–22 ms; Qwen step 41–79 ms	21.16 / 21.51 / 21.70; 40.64 / 60.18 / 79.37	scripts/phase6_analysis.py step_ms_alone	ok (40.64 rounds to 41)
Fig. 9: Qwen γ=2 0.94 accepted, 70 tok/s; lookup γ=6 0.45, 161 tok/s	0.9374 / 70.29; 0.4531 / 161.06	3e38251 code edits, greedy	ok
lookup 6 / model 2: lower acceptance and faster on every set; +8% mixed	8/8 and 8/8 set×mode cells; 1.0771300880319867	fdd3be9	ok
0.3.0 README acceptance lower than 0.2.0, throughput higher	0.7674 → 0.3920; 73.49 → 84.15 tok/s	8d036d7, 3c43e08 HumanEval greedy	ok
Qwen2.5-Instruct defaults top_k 20, top_p 0.8, repetition_penalty 1.1	20, 0.8, 1.1	generation_config.json	ok
Table: 0.2.0 20/20; Phase 6 width-3/5 all; width-7 54/60; 0.3.0 17/20	20/20; 60/60 for all 5 engines; 54/60 for all 4; 17/20	greedy_token_match_compiled	ok
Width-7 methods produced bit-identical outputs	sha256 equal on 60/60 greedy prompts	3e38251 outputs	ok
Fig. 5: 0.95 mixed, 0.76 code edits; HumanEval 0.9505	0.9508431825338723, 0.7632609217619635, 0.9505080466064945	3e38251	ok
about 1,400 steps; 75% off the best arm	1425; 0.7529824561403509	3e38251 arm report, analysis	ok
re-pricing −1% to +5%	0.9892836322547345–1.0477376649698797	analysis counterfactual_over_measured	ok
reward overstates composite 1.24×–1.28×	1.241774272171092, 1.2807068146451581	analysis reward_over_tps	ok
best arm changed 16 of 29; per-prompt oracle ~8%	16 / 29; 1.0802380741939623	analysis prompt_oracles	ok
distribution test at γ=4, batch 10,000	_ENGINE_N = 10_000, SamplingConfig(gamma=4)	tests/integration/test_output_distribution.py	ok
169 tests failed or errored after the default switch	150 failed + 19 errors in that session's run	a test-run count, not a result JSON	ok (not from a JSON)

One mismatch, left as written for you to decide: "parameter ratio (0.33)" in lesson 2. The real ratio is 0.32. Suggested
wording, if you want it: "…its cost ratio (0.90) tracks the layer ratio (0.86), not the parameter ratio (0.32)."

Gate

Local, on this branch:

• make lint (ruff incl. scripts/article, format, import-linter), mypy --strict flashspec: clean
• pytest tests/unit tests/integration -m "not gpu": 561 passed, 5 skipped (coverage 88.92%), including test_article_figures.py
• mkdocs build: the Engineering notes page renders with all 9 images; 30 warnings, the same griffe docstring warnings as main
• CI on this PR: docs-build pass, lint pass, onnx-parity pass, paper pass, test-chaos pass, test-cpu pass