Skip to content

Commit a27bcbf

Browse files
committed
docs(bench): the schedule column — the default-only table understated mcpp badly
你问得对:上一版表是 **`bmi_schedule` 关着**测的,而那正是专门优化冷构建与级联的 开关。补测同一棵树、同样五个场景的 `+schedule=on` 对照臂,**10 ok / 0 failed**: | 场景 | schedule=on | 默认 | 对 cmake | |---|---|---|---| | cold | **37.56s** | 92.49s | 1.29x → **3.18x** | | touch-hub | **1.04s** | 1.79s | 54.96x → **93.93x** | | edit-body | **29.65s** | 88.38s | 1.11x → **3.32x** | | edit-comment | **30.44s** | 93.81s | 1.04x → **3.22x** | | noop | 0.72s | 0.74s | 0.49x → 0.50x(仍然输) | **我先前对 `edit-body`/`edit-comment` 的解释只对默认配置成立。** 我写的是「级联是 欠的,所以 mcpp 没有优势」—— 级联确实是欠的,但开了调度之后 mcpp 把**同一份欠下的 工作**做快了 3.3 倍(BMI 一产出就发布,而不是等代码生成结束)。只放默认列,读者会 得出「mcpp 在真实编辑场景上和 cmake 一样」的结论,那是错的。 两列并排,因为它们回答不同的问题:默认列是用户今天拿到的,调度列是这个开关能给 什么。`noop` 两列都输,照写。 **§8b 的正确性缺陷在这里没有复现**(十个格子全 ok)。它只在生成的 fixture 上出现 —— 那条紧密的 unit_0→unit_1 链会撞进窗口。该键因此仍然默认关闭:只有一个工作负载 能暴露的缺陷,仍然是缺陷。
1 parent 88fc870 commit a27bcbf

3 files changed

Lines changed: 214 additions & 14 deletions

File tree

bench/README.md

Lines changed: 26 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -200,13 +200,32 @@ First measurement in which all three arms produce a running binary — the cmake
200200
and xmake columns below were `failed` cells until the arms were finished, and the
201201
table that stood here was mcpp-against-mcpp for that reason.
202202

203-
| scenario | **mcpp** | cmake | xmake |
204-
|---|---|---|---|
205-
| `cold` | **92.49s** · 1.29x | 119.46s · 1.00x | 105.02s · 1.14x |
206-
| `noop` | **0.74s** · 0.49x | 0.36s · 1.00x | 0.40s · 0.90x |
207-
| `touch-hub` | **1.79s** · 54.96x | 98.16s · 1.00x | 98.16s · 1.00x |
208-
| `edit-body` | **88.38s** · 1.11x | 98.43s · 1.00x | 98.00s · 1.00x |
209-
| `edit-comment` | **93.81s** · 1.04x | 97.98s · 1.00x | 97.97s · 1.00x |
203+
| scenario | **mcpp** `bmi_schedule=on` | mcpp default | cmake | xmake |
204+
|---|---|---|---|---|
205+
| `cold` | **37.56s** · 3.18x | 92.49s · 1.29x | 119.46s · 1.00x | 105.02s · 1.14x |
206+
| `noop` | **0.72s** · 0.50x | 0.74s · 0.49x | 0.36s · 1.00x | 0.40s · 0.90x |
207+
| `touch-hub` | **1.04s** · 93.93x | 1.79s · 54.96x | 98.16s · 1.00x | 98.16s · 1.00x |
208+
| `edit-body` | **29.65s** · 3.32x | 88.38s · 1.11x | 98.43s · 1.00x | 98.00s · 1.00x |
209+
| `edit-comment` | **30.44s** · 3.22x | 93.81s · 1.04x | 97.98s · 1.00x | 97.97s · 1.00x |
210+
211+
**Both mcpp columns are here because one of them was misleading on its own.**
212+
The default column is what a user gets today; `bmi_schedule=on` is the opt-in
213+
split schedule, and leaving it out understated mcpp badly — `edit-body` reads
214+
1.11x in the default column and 3.32x with the schedule on.
215+
216+
* **`edit-body` and `edit-comment` are not "no advantage".** The cascade really
217+
is owed in both (the perturbed function body lives in an interface unit, so
218+
the BMI genuinely changes). The default column shows mcpp doing that owed work
219+
at cmake's pace; the schedule column shows it doing the SAME work 3.3x faster,
220+
by publishing each BMI as soon as it exists instead of after code generation.
221+
* **`noop` is the one mcpp loses outright**, in both columns: 0.72–0.74s against
222+
cmake's 0.36s. That is per-invocation overhead, and it is the number a user
223+
feels on every edit-build cycle.
224+
* **The `bmi_schedule` correctness bug (§8b) does NOT reproduce here.** All ten
225+
cells are `ok`. It reproduces on the generated fixture, whose tight
226+
unit_0→unit_1 chain hits the window; three real trees (mcpp's own and both
227+
xlings styles) do not. That is why the key is still opt-in — a defect that
228+
only one workload can show is still a defect.
210229

211230
<sub>xlings `2026.8.11.2`, gcc 16.1.0 payload, Linux x86_64 · i9-13900K · n=1 ·
212231
`--baseline cmake`. Raw report: `bench/results/xlings-3way-20260814/`.</sub>

bench/README.zh-CN.md

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -164,13 +164,28 @@ job 一个测量都没有,这个状态持续了好几周。
164164
**第一次三条臂都能产出可运行二进制的测量** —— 在此之前 cmake 与 xmake 两列一直是
165165
`failed`,所以这里原本只有 mcpp 对 mcpp。
166166

167-
| 场景 | **mcpp** | cmake | xmake |
168-
|---|---|---|---|
169-
| `cold` | **92.49s** · 1.29x | 119.46s · 1.00x | 105.02s · 1.14x |
170-
| `noop` | **0.74s** · 0.49x | 0.36s · 1.00x | 0.40s · 0.90x |
171-
| `touch-hub` | **1.79s** · 54.96x | 98.16s · 1.00x | 98.16s · 1.00x |
172-
| `edit-body` | **88.38s** · 1.11x | 98.43s · 1.00x | 98.00s · 1.00x |
173-
| `edit-comment` | **93.81s** · 1.04x | 97.98s · 1.00x | 97.97s · 1.00x |
167+
| 场景 | **mcpp** `bmi_schedule=on` | mcpp 默认 | cmake | xmake |
168+
|---|---|---|---|---|
169+
| `cold` | **37.56s** · 3.18x | 92.49s · 1.29x | 119.46s · 1.00x | 105.02s · 1.14x |
170+
| `noop` | **0.72s** · 0.50x | 0.74s · 0.49x | 0.36s · 1.00x | 0.40s · 0.90x |
171+
| `touch-hub` | **1.04s** · 93.93x | 1.79s · 54.96x | 98.16s · 1.00x | 98.16s · 1.00x |
172+
| `edit-body` | **29.65s** · 3.32x | 88.38s · 1.11x | 98.43s · 1.00x | 98.00s · 1.00x |
173+
| `edit-comment` | **30.44s** · 3.22x | 93.81s · 1.04x | 97.98s · 1.00x | 97.97s · 1.00x |
174+
175+
**两列 mcpp 都在这里,是因为只放一列会误导。** 默认列是用户今天拿到的东西;
176+
`bmi_schedule=on` 是可选的分离调度,不放它会严重低估 mcpp —— `edit-body` 在默认
177+
列是 1.11x,开了调度是 3.32x。
178+
179+
* **`edit-body``edit-comment` 不是「没有优势」。** 两者的级联都是**欠的**
180+
(被扰动的函数体在接口单元里,BMI 真的变了)。默认列显示 mcpp 以 cmake 的速度
181+
做完这份欠下的工作;调度列显示它把**同一份工作**做快了 3.3 倍 —— 靠的是 BMI
182+
一产出就发布,而不是等代码生成结束。
183+
* **`noop` 是 mcpp 真正输的一项**,两列都输:0.72–0.74s 对 cmake 的 0.36s。这是
184+
每次调用的固定开销,也是用户在每一次「改一行、构建一次」里都感受得到的数字。
185+
* **§8b 那个 `bmi_schedule` 正确性缺陷在这里没有复现**,十个格子全 `ok`。它只在
186+
生成的 fixture 上复现 —— 那条紧密的 `unit_0→unit_1` 链会撞进窗口;三棵真实的树
187+
(mcpp 自己和 xlings 两种风格)都不会。这也正是该键仍然默认关闭的原因:**只有
188+
一个工作负载能暴露的缺陷,仍然是缺陷。**
174189

175190
<sub>xlings `2026.8.11.2`,gcc 16.1.0 载荷,Linux x86_64 · i9-13900K · n=1 ·
176191
`--baseline cmake`。原始报告:`bench/results/xlings-3way-20260814/`。</sub>
Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
{
2+
"protocol_version": 1,
3+
"started_at": "2026-08-13T22:29:14Z",
4+
"host": {
5+
"os": "linux",
6+
"arch": "x86_64",
7+
"cpu_model": "13th Gen Intel(R) Core(TM) i9-13900K",
8+
"logical_cores": 32,
9+
"physical_cores": 24,
10+
"heterogeneous": true,
11+
"ram_bytes": 67147722752,
12+
"toolchain": "/home/speak/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin/g++"
13+
},
14+
"cells": [
15+
{
16+
"engine": "mcpp@2026.8.13.1",
17+
"compiler": "gcc",
18+
"profile": "release",
19+
"scenario": "cold",
20+
"fixture": "xlings-2026.8.11.2",
21+
"variant": "modules",
22+
"status": "ok",
23+
"note": "mcpp 2026.8.13.1",
24+
"runs": 1,
25+
"median_s": 91.608,
26+
"min_s": 91.608,
27+
"max_s": 91.608,
28+
"samples": [91.608]
29+
},
30+
{
31+
"engine": "mcpp@2026.8.13.1",
32+
"compiler": "gcc",
33+
"profile": "release",
34+
"scenario": "noop",
35+
"fixture": "xlings-2026.8.11.2",
36+
"variant": "modules",
37+
"status": "ok",
38+
"note": "mcpp 2026.8.13.1",
39+
"runs": 1,
40+
"median_s": 0.723,
41+
"min_s": 0.723,
42+
"max_s": 0.723,
43+
"samples": [0.723]
44+
},
45+
{
46+
"engine": "mcpp@2026.8.13.1",
47+
"compiler": "gcc",
48+
"profile": "release",
49+
"scenario": "touch-hub",
50+
"fixture": "xlings-2026.8.11.2",
51+
"variant": "modules",
52+
"status": "ok",
53+
"note": "mcpp 2026.8.13.1",
54+
"runs": 1,
55+
"median_s": 1.786,
56+
"min_s": 1.786,
57+
"max_s": 1.786,
58+
"samples": [1.786]
59+
},
60+
{
61+
"engine": "mcpp@2026.8.13.1",
62+
"compiler": "gcc",
63+
"profile": "release",
64+
"scenario": "edit-body",
65+
"fixture": "xlings-2026.8.11.2",
66+
"variant": "modules",
67+
"status": "ok",
68+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
69+
"runs": 1,
70+
"median_s": 89.457,
71+
"min_s": 89.457,
72+
"max_s": 89.457,
73+
"samples": [89.457]
74+
},
75+
{
76+
"engine": "mcpp@2026.8.13.1",
77+
"compiler": "gcc",
78+
"profile": "release",
79+
"scenario": "edit-comment",
80+
"fixture": "xlings-2026.8.11.2",
81+
"variant": "modules",
82+
"status": "ok",
83+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
84+
"runs": 1,
85+
"median_s": 93.843,
86+
"min_s": 93.843,
87+
"max_s": 93.843,
88+
"samples": [93.843]
89+
},
90+
{
91+
"engine": "mcpp@2026.8.13.1+schedule=on",
92+
"compiler": "gcc",
93+
"profile": "release",
94+
"scenario": "cold",
95+
"fixture": "xlings-2026.8.11.2",
96+
"variant": "modules",
97+
"status": "ok",
98+
"note": "mcpp 2026.8.13.1",
99+
"runs": 1,
100+
"median_s": 37.561,
101+
"min_s": 37.561,
102+
"max_s": 37.561,
103+
"samples": [37.561]
104+
},
105+
{
106+
"engine": "mcpp@2026.8.13.1+schedule=on",
107+
"compiler": "gcc",
108+
"profile": "release",
109+
"scenario": "noop",
110+
"fixture": "xlings-2026.8.11.2",
111+
"variant": "modules",
112+
"status": "ok",
113+
"note": "mcpp 2026.8.13.1",
114+
"runs": 1,
115+
"median_s": 0.723,
116+
"min_s": 0.723,
117+
"max_s": 0.723,
118+
"samples": [0.723]
119+
},
120+
{
121+
"engine": "mcpp@2026.8.13.1+schedule=on",
122+
"compiler": "gcc",
123+
"profile": "release",
124+
"scenario": "touch-hub",
125+
"fixture": "xlings-2026.8.11.2",
126+
"variant": "modules",
127+
"status": "ok",
128+
"note": "mcpp 2026.8.13.1",
129+
"runs": 1,
130+
"median_s": 1.045,
131+
"min_s": 1.045,
132+
"max_s": 1.045,
133+
"samples": [1.045]
134+
},
135+
{
136+
"engine": "mcpp@2026.8.13.1+schedule=on",
137+
"compiler": "gcc",
138+
"profile": "release",
139+
"scenario": "edit-body",
140+
"fixture": "xlings-2026.8.11.2",
141+
"variant": "modules",
142+
"status": "ok",
143+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
144+
"runs": 1,
145+
"median_s": 29.646,
146+
"min_s": 29.646,
147+
"max_s": 29.646,
148+
"samples": [29.646]
149+
},
150+
{
151+
"engine": "mcpp@2026.8.13.1+schedule=on",
152+
"compiler": "gcc",
153+
"profile": "release",
154+
"scenario": "edit-comment",
155+
"fixture": "xlings-2026.8.11.2",
156+
"variant": "modules",
157+
"status": "ok",
158+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
159+
"runs": 1,
160+
"median_s": 30.444,
161+
"min_s": 30.444,
162+
"max_s": 30.444,
163+
"samples": [30.444]
164+
}
165+
]
166+
}

0 commit comments

Comments
 (0)