Skip to content

Commit a413bca

Browse files
committed
docs(bench): the split tree, where the refactor beats the engine choice
split 树同样 15 ok / 0 failed / 零告警。两种代码风格现在都有真实三方数据。 | 场景 | mcpp | cmake | xmake | |---|---|---|---| | cold | 27.59s · 1.82x | 50.13s | 41.90s · 1.20x | | noop | 0.79s · **0.44x** | 0.34s | 0.50s · 0.68x | | touch-hub | **1.32s · 20.08x** | 26.60s | 31.68s · 0.84x | | edit-body | 1.79s · **0.75x** | 1.35s | 1.49s · 0.91x | | edit-comment | 24.17s · 1.09x | 26.35s | 31.28s · 0.84x | **最重要的一条不是 mcpp 赢在哪,而是重构本身的收益盖过了引擎选择**:mcpp 的 cold 从 92.49s(combined)降到 27.59s(3.35x),而 cmake 自己也从 119.46s 降到 50.13s (2.38x)。把实现搬出接口单元,比换构建工具更值 —— 这是这两个 pin 存在的意义, 也是这套件能给用户的最有用的一句话。 **`edit-body` 0.75x —— 这里 mcpp 比 cmake 慢。** 函数体在 `.cpp` 里,只重编一个 object、没有级联,于是场景量的是每次调用的固定开销而非图推理,和 `noop` 是同一笔 成本。在这条轴上 mcpp 没有优势,数字就该这么写。 `touch-hub` 20.08x 比 combined 的 54.96x 小,因为下游本来就没剩多少可跳的了。
1 parent 6c24d92 commit a413bca

3 files changed

Lines changed: 297 additions & 0 deletions

File tree

bench/README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,35 @@ mcpp LOSING:
226226
cmake and xmake decide by timestamp and rebuild all of them — to within 0.00s
227227
of each other, which is what two timestamp-driven engines should look like.
228228

229+
230+
#### The same three engines on the SPLIT tree
231+
232+
Same project, implementations moved out of the interface units. This is the axis
233+
the two pins exist for, and it changes the answer more than the engine does.
234+
235+
| scenario | **mcpp** | cmake | xmake |
236+
|---|---|---|---|
237+
| `cold` | **27.59s** · 1.82x | 50.13s · 1.00x | 41.90s · 1.20x |
238+
| `noop` | **0.79s** · 0.44x | 0.34s · 1.00x | 0.50s · 0.68x |
239+
| `touch-hub` | **1.32s** · 20.08x | 26.60s · 1.00x | 31.68s · 0.84x |
240+
| `edit-body` | **1.79s** · 0.75x | 1.35s · 1.00x | 1.49s · 0.91x |
241+
| `edit-comment` | **24.17s** · 1.09x | 26.35s · 1.00x | 31.28s · 0.84x |
242+
243+
<sub>xlings `2026.8.13.1`, `modules-impl`, same host and payload as above. Raw
244+
report: `bench/results/xlings-3way-20260814/xlings-split-3way.json`.</sub>
245+
246+
* **The refactor beats every engine choice on this workload.** `cold` falls from
247+
92.49s to 27.59s for mcpp — 3.35x — and cmake's own cold falls 119.46s → 50.13s
248+
(2.38x). Moving implementations out of interface units buys more than switching
249+
build tool does.
250+
* **`edit-body` is 0.75x — mcpp is SLOWER than cmake here**, 1.79s against 1.35s.
251+
With the body in a `.cpp`, one object recompiles and nothing cascades, so the
252+
scenario measures per-invocation overhead rather than graph reasoning — the
253+
same fixed cost `noop` shows. On this axis mcpp has no advantage to offer and
254+
the number says so.
255+
* **`touch-hub` still pays: 20.08x.** Smaller than the combined tree's 54.96x
256+
because there is simply less downstream work left to skip.
257+
229258
#### The two code styles, mcpp against mcpp
230259

231260
Older run, kept because it is the only side-by-side of the two pinned

bench/README.zh-CN.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,33 @@ job 一个测量都没有,这个状态持续了好几周。
187187
上一份比,跳过 45 个导入者;cmake 与 xmake 按时间戳判断,把它们全部重建 ——
188188
两者相差 0.00s,这正是两个时间戳驱动的引擎该有的样子。
189189

190+
191+
## 4c. 同样三个引擎在 split 树上
192+
193+
同一个工程,实现从接口单元里搬了出去。这正是两个 pin 存在的意义 —— 而它对结果的
194+
影响**比换构建工具更大**
195+
196+
| 场景 | **mcpp** | cmake | xmake |
197+
|---|---|---|---|
198+
| `cold` | **27.59s** · 1.82x | 50.13s · 1.00x | 41.90s · 1.20x |
199+
| `noop` | **0.79s** · 0.44x | 0.34s · 1.00x | 0.50s · 0.68x |
200+
| `touch-hub` | **1.32s** · 20.08x | 26.60s · 1.00x | 31.68s · 0.84x |
201+
| `edit-body` | **1.79s** · 0.75x | 1.35s · 1.00x | 1.49s · 0.91x |
202+
| `edit-comment` | **24.17s** · 1.09x | 26.35s · 1.00x | 31.28s · 0.84x |
203+
204+
<sub>xlings `2026.8.13.1`,`modules-impl`,主机与载荷同上。原始报告:
205+
`bench/results/xlings-3way-20260814/xlings-split-3way.json`。</sub>
206+
207+
* **在这个工作负载上,重构的收益盖过了任何引擎选择。** mcpp 的 `cold` 从 92.49s
208+
降到 27.59s(3.35x),而 cmake 自己的 cold 也从 119.46s 降到 50.13s(2.38x)。
209+
把实现搬出接口单元,比换构建工具更值。
210+
* **`edit-body` 是 0.75x —— 这里 mcpp 比 cmake 慢**(1.79s 对 1.35s)。函数体在
211+
`.cpp` 里,只重编一个 object、没有级联,于是这个场景量的是**每次调用的固定
212+
开销**而不是图推理 —— 和 `noop` 反映的是同一笔成本。在这条轴上 mcpp 没有优势
213+
可言,数字也就这么写。
214+
* **`touch-hub` 仍然值:20.08x。** 比 combined 树的 54.96x 小,因为下游本来就
215+
没剩多少可跳的了。
216+
190217
## 5. 可观测性:跑的时候看得见
191218

192219
harness 会把进度实时打到 **stderr**(逐行 flush),stdout 留给报告:
Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
{
2+
"protocol_version": 1,
3+
"started_at": "2026-08-13T18:19:51Z",
4+
"host": {
5+
"os": "linux",
6+
"arch": "x86_64",
7+
"cpu_model": "13th Gen Intel(R) Core(TM) i9-13900K",
8+
"logical_cores": 32,
9+
"physical_cores": 24,
10+
"heterogeneous": true,
11+
"ram_bytes": 67147722752,
12+
"toolchain": "/home/speak/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin/g++"
13+
},
14+
"cells": [
15+
{
16+
"engine": "mcpp@2026.8.13.1",
17+
"compiler": "gcc",
18+
"profile": "release",
19+
"scenario": "cold",
20+
"fixture": "xlings-2026.8.13.1",
21+
"variant": "modules-impl",
22+
"status": "ok",
23+
"note": "mcpp 2026.8.13.1",
24+
"runs": 1,
25+
"median_s": 27.588,
26+
"min_s": 27.588,
27+
"max_s": 27.588,
28+
"samples": [27.588]
29+
},
30+
{
31+
"engine": "mcpp@2026.8.13.1",
32+
"compiler": "gcc",
33+
"profile": "release",
34+
"scenario": "noop",
35+
"fixture": "xlings-2026.8.13.1",
36+
"variant": "modules-impl",
37+
"status": "ok",
38+
"note": "mcpp 2026.8.13.1",
39+
"runs": 1,
40+
"median_s": 0.785,
41+
"min_s": 0.785,
42+
"max_s": 0.785,
43+
"samples": [0.785]
44+
},
45+
{
46+
"engine": "mcpp@2026.8.13.1",
47+
"compiler": "gcc",
48+
"profile": "release",
49+
"scenario": "touch-hub",
50+
"fixture": "xlings-2026.8.13.1",
51+
"variant": "modules-impl",
52+
"status": "ok",
53+
"note": "mcpp 2026.8.13.1",
54+
"runs": 1,
55+
"median_s": 1.325,
56+
"min_s": 1.325,
57+
"max_s": 1.325,
58+
"samples": [1.325]
59+
},
60+
{
61+
"engine": "mcpp@2026.8.13.1",
62+
"compiler": "gcc",
63+
"profile": "release",
64+
"scenario": "edit-body",
65+
"fixture": "xlings-2026.8.13.1",
66+
"variant": "modules-impl",
67+
"status": "ok",
68+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
69+
"runs": 1,
70+
"median_s": 1.790,
71+
"min_s": 1.790,
72+
"max_s": 1.790,
73+
"samples": [1.790]
74+
},
75+
{
76+
"engine": "mcpp@2026.8.13.1",
77+
"compiler": "gcc",
78+
"profile": "release",
79+
"scenario": "edit-comment",
80+
"fixture": "xlings-2026.8.13.1",
81+
"variant": "modules-impl",
82+
"status": "ok",
83+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
84+
"runs": 1,
85+
"median_s": 24.168,
86+
"min_s": 24.168,
87+
"max_s": 24.168,
88+
"samples": [24.168]
89+
},
90+
{
91+
"engine": "cmake",
92+
"compiler": "gcc",
93+
"profile": "release",
94+
"scenario": "cold",
95+
"fixture": "xlings-2026.8.13.1",
96+
"variant": "modules-impl",
97+
"status": "ok",
98+
"note": "cmake version 4.0.2 + ninja",
99+
"runs": 1,
100+
"median_s": 50.129,
101+
"min_s": 50.129,
102+
"max_s": 50.129,
103+
"samples": [50.129]
104+
},
105+
{
106+
"engine": "cmake",
107+
"compiler": "gcc",
108+
"profile": "release",
109+
"scenario": "noop",
110+
"fixture": "xlings-2026.8.13.1",
111+
"variant": "modules-impl",
112+
"status": "ok",
113+
"note": "cmake version 4.0.2 + ninja",
114+
"runs": 1,
115+
"median_s": 0.342,
116+
"min_s": 0.342,
117+
"max_s": 0.342,
118+
"samples": [0.342]
119+
},
120+
{
121+
"engine": "cmake",
122+
"compiler": "gcc",
123+
"profile": "release",
124+
"scenario": "touch-hub",
125+
"fixture": "xlings-2026.8.13.1",
126+
"variant": "modules-impl",
127+
"status": "ok",
128+
"note": "cmake version 4.0.2 + ninja",
129+
"runs": 1,
130+
"median_s": 26.600,
131+
"min_s": 26.600,
132+
"max_s": 26.600,
133+
"samples": [26.600]
134+
},
135+
{
136+
"engine": "cmake",
137+
"compiler": "gcc",
138+
"profile": "release",
139+
"scenario": "edit-body",
140+
"fixture": "xlings-2026.8.13.1",
141+
"variant": "modules-impl",
142+
"status": "ok",
143+
"note": "cmake version 4.0.2 + ninja · perturbation: in-body",
144+
"runs": 1,
145+
"median_s": 1.349,
146+
"min_s": 1.349,
147+
"max_s": 1.349,
148+
"samples": [1.349]
149+
},
150+
{
151+
"engine": "cmake",
152+
"compiler": "gcc",
153+
"profile": "release",
154+
"scenario": "edit-comment",
155+
"fixture": "xlings-2026.8.13.1",
156+
"variant": "modules-impl",
157+
"status": "ok",
158+
"note": "cmake version 4.0.2 + ninja · perturbation: in-body",
159+
"runs": 1,
160+
"median_s": 26.349,
161+
"min_s": 26.349,
162+
"max_s": 26.349,
163+
"samples": [26.349]
164+
},
165+
{
166+
"engine": "xmake",
167+
"compiler": "gcc",
168+
"profile": "release",
169+
"scenario": "cold",
170+
"fixture": "xlings-2026.8.13.1",
171+
"variant": "modules-impl",
172+
"status": "ok",
173+
"note": "xmake v3.1.0+HEAD.96ad28e, A cross-platform build utility based on Lua",
174+
"runs": 1,
175+
"median_s": 41.904,
176+
"min_s": 41.904,
177+
"max_s": 41.904,
178+
"samples": [41.904]
179+
},
180+
{
181+
"engine": "xmake",
182+
"compiler": "gcc",
183+
"profile": "release",
184+
"scenario": "noop",
185+
"fixture": "xlings-2026.8.13.1",
186+
"variant": "modules-impl",
187+
"status": "ok",
188+
"note": "xmake v3.1.0+HEAD.96ad28e, A cross-platform build utility based on Lua",
189+
"runs": 1,
190+
"median_s": 0.502,
191+
"min_s": 0.502,
192+
"max_s": 0.502,
193+
"samples": [0.502]
194+
},
195+
{
196+
"engine": "xmake",
197+
"compiler": "gcc",
198+
"profile": "release",
199+
"scenario": "touch-hub",
200+
"fixture": "xlings-2026.8.13.1",
201+
"variant": "modules-impl",
202+
"status": "ok",
203+
"note": "xmake v3.1.0+HEAD.96ad28e, A cross-platform build utility based on Lua",
204+
"runs": 1,
205+
"median_s": 31.676,
206+
"min_s": 31.676,
207+
"max_s": 31.676,
208+
"samples": [31.676]
209+
},
210+
{
211+
"engine": "xmake",
212+
"compiler": "gcc",
213+
"profile": "release",
214+
"scenario": "edit-body",
215+
"fixture": "xlings-2026.8.13.1",
216+
"variant": "modules-impl",
217+
"status": "ok",
218+
"note": "xmake v3.1.0+HEAD.96ad28e, A cross-platform build utility based on Lua · perturbation: in-body",
219+
"runs": 1,
220+
"median_s": 1.486,
221+
"min_s": 1.486,
222+
"max_s": 1.486,
223+
"samples": [1.486]
224+
},
225+
{
226+
"engine": "xmake",
227+
"compiler": "gcc",
228+
"profile": "release",
229+
"scenario": "edit-comment",
230+
"fixture": "xlings-2026.8.13.1",
231+
"variant": "modules-impl",
232+
"status": "ok",
233+
"note": "xmake v3.1.0+HEAD.96ad28e, A cross-platform build utility based on Lua · perturbation: in-body",
234+
"runs": 1,
235+
"median_s": 31.281,
236+
"min_s": 31.281,
237+
"max_s": 31.281,
238+
"samples": [31.281]
239+
}
240+
]
241+
}

0 commit comments

Comments
 (0)