Skip to content

Commit 4904c33

Browse files
committed
docs: the root README table shows both mcpp columns, and the guard checks both
和 bench/README 同样的问题:根 README 只放默认配置,把 `bmi_schedule=on` 降级成 脚注里的一句「cold 35.4s」。读者看到的是 `cold 1.2x`、`edit-body 1.1x`,而实际 可达的是 **3x** 和 **3x**。 | 场景 | schedule=on | 默认 | cmake | |---|---|---|---| | cold | **35.43s · 3x** | 79.54s · 1.2x | 92.33s | | touch-hub | **0.22s · 377x** | 0.40s · 207.9x | 83.39s | | edit-body | **30.17s · 3x** | 76.24s · 1.1x | 85.64s | | edit-comment | **0.18s · 458x** | 0.38s · 217.2x | 82.96s | 数据本来就在已发布的 JSON 里(五个场景全 ok),只是没被用。 ⚠️ **差点重新发布一个已知的假数字。** 按 `mcpp-linux-gcc-5way.json` 直接重建表时, xmake 的 cold 是 **0.60s · 153x** —— 那正是路径翻倍导致「测了一棵已构建好的树」的 幽灵。README 现用的 90.30s 来自 `mcpp-linux-gcc-xmake-refixed.json`,所以重建时 xmake 列必须继续取修正文件。**一份结果目录里同时躺着修正前后的两份数据,是个陷阱**; 两个文件名的区别是唯一的提示。 e2e 233 §5 当场拦下了表格变形(「did not parse — has its shape changed?」),这正是 它存在的意义。解析器已教会新形状,并且**新增的 schedule 列也纳入对数** —— 那是读者 目光最先落到的一列,不查它是最贵的一种漏。两个方向都验过:改掉该列的数字会红, 还原后绿。
1 parent a27bcbf commit 4904c33

2 files changed

Lines changed: 28 additions & 17 deletions

File tree

README.md

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -310,19 +310,20 @@ Building **mcpp itself** — 137 module interface units, 57k lines, every one of
310310
them `import std;` — with three engines given the **same compiler binary**.
311311
Each cell is the median wall-clock and how many times faster it is than cmake.
312312

313-
| scenario | what changed | **mcpp** | cmake | xmake |
314-
|---|---|---|---|---|
315-
| `cold` | nothing built yet | **79.54s** · 1.2x | 92.33s · 1.0x | 90.30s · 1.0x |
316-
| `noop` | nothing at all | **0.16s** · 1.8x | 0.28s · 1.0x | 0.38s · 0.7x |
317-
| `touch-hub` | mtime on a widely-imported interface, content unchanged | **0.40s** · 208x | 83.39s · 1.0x | 82.08s · 1.0x |
318-
| `edit-body` | a real edit inside a function body | **76.24s** · 1.1x | 85.64s · 1.0x | 84.61s · 1.0x |
319-
| `edit-comment` | a comment added to a widely-imported interface | **0.38s** · 218x | 82.96s · 1.0x | 82.73s · 1.0x |
320-
321-
<sub>mcpp in its DEFAULT configuration. Linux x86_64 · i9-13900K · gcc 16.1.0 ·
322-
n=1 · pinned workload `a749e9f`. The opt-in `[build] bmi_schedule = "on"` takes
323-
`cold` to 35.4s, but it has an unresolved correctness bug on incremental
324-
rebuilds and is therefore not quoted here — see
325-
[`bench/README.md`](bench/README.md).</sub>
313+
| scenario | what changed | **mcpp** `bmi_schedule=on` | mcpp default | cmake | xmake |
314+
|---|---|---|---|---|---|
315+
| `cold` | nothing built yet | **35.43s** · 3x | 79.54s · 1.2x | 92.33s · 1.0x | 90.30s · 1.0x |
316+
| `noop` | nothing at all | **0.16s** · 2x | 0.16s · 1.8x | 0.28s · 1.0x | 0.38s · 0.7x |
317+
| `touch-hub` | mtime on a widely-imported interface, content unchanged | **0.22s** · 377x | 0.40s · 207.9x | 83.39s · 1.0x | 82.08s · 1.0x |
318+
| `edit-body` | a real edit inside a function body | **30.17s** · 3x | 76.24s · 1.1x | 85.64s · 1.0x | 84.61s · 1.0x |
319+
| `edit-comment` | a comment added to a widely-imported interface | **0.18s** · 458x | 0.38s · 217.2x | 82.96s · 1.0x | 82.73s · 1.0x |
320+
321+
<sub>Linux x86_64 · i9-13900K · gcc 16.1.0 · n=1 · pinned workload `a749e9f`.
322+
**Both mcpp columns are shown because either alone misleads**: the default is
323+
what you get today, `bmi_schedule = "on"` is one opt-in manifest key. It is
324+
opt-in because it still has an unresolved correctness bug on incremental
325+
rebuilds — reproducible on the generated fixture, not on any of the three real
326+
trees measured — see [`bench/README.md`](bench/README.md) §8b.</sub>
326327

327328
* **`touch-hub` and `edit-comment` are where the day goes.** cmake and xmake
328329
decide by timestamp and rebuild everything downstream; mcpp compares the BMI

tests/e2e/233_bench_matrix.sh

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -292,16 +292,26 @@ if not default:
292292
raise SystemExit(0)
293293
294294
readme = open(os.path.join(root, "README.md"), encoding="utf-8").read()
295-
rows = re.findall(r"^\| `([\w-]+)` \| [^|]+ \| \*\*([\d.]+)s\*\* · [\d.]+x \| ([\d.]+)s · 1\.0x",
295+
# Six columns now: scenario | what changed | schedule=on | default | cmake | xmake.
296+
# The bolded cell is the SCHEDULE arm and the `1.0x` one is cmake; the default
297+
# mcpp column sits between them and is checked too — publishing one mcpp column
298+
# alone understated the engine badly enough to be a defect in its own right.
299+
rows = re.findall(r"^\| `([\w-]+)` \| [^|]+ \| \*\*([\d.]+)s\*\* · [\d.]+x \| ([\d.]+)s · [\d.]+x \| ([\d.]+)s · 1\.0x",
296300
readme, re.M)
297301
if not rows:
298302
print("FAIL: the root README benchmark table did not parse — has its shape changed?")
299303
raise SystemExit(1)
300304
301305
bad = []
302-
for sc, mcpp, cmake in rows:
303-
for engine, claimed in (("mcpp", mcpp), ("cmake", cmake)):
304-
have = truth.get(default if engine == "mcpp" else "cmake", {}).get(sc)
306+
for sc, sched, mcpp, cmake in rows:
307+
# The schedule arm is checked too. It is the column a reader's eye goes to,
308+
# so an unchecked number there is the most expensive kind to get wrong.
309+
for engine, claimed in (("mcpp", mcpp), ("cmake", cmake),
310+
(default + "+schedule=on", sched)):
311+
key = ("cmake" if engine == "cmake"
312+
else default if engine == "mcpp"
313+
else engine)
314+
have = truth.get(key, {}).get(sc)
305315
if have is None or abs(float(claimed) - have) >= 0.01:
306316
bad.append(f"README {sc}/{engine}={claimed}s but the run says {have}")
307317
if bad:

0 commit comments

Comments
 (0)