Skip to content

Commit ce8d32b

Browse files
voidfilesvoidfiles
committed
fix: stabilize and publish latest benchmarks
1 parent 354bbe9 commit ce8d32b

9 files changed

Lines changed: 427 additions & 3 deletions

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,11 @@ Correctness is a gate, not an assumption. Before timing an adapter, the runner v
3636

3737
## Canonical results
3838

39-
The versioned 2026-09-06 snapshot is available as separate [primitive](results/reports/2026-09-06-cpython-3.14-primitive.md) and [encoded](results/reports/2026-09-06-cpython-3.14-encoded.md) reports. These are machine-specific measurements, not a portable performance baseline.
39+
The versioned 2026-09-13 snapshot is available as separate [primitive](results/reports/2026-09-13-cpython-3.14-primitive.md) and [encoded](results/reports/2026-09-13-cpython-3.14-encoded.md) reports. These are machine-specific measurements, not a portable performance baseline.
4040

41-
> **Comparison warning:** Results from different machines or runs are not directly comparable. `pyperf` flagged 37 of 38 primitive benchmarks and 44 of 48 encoded benchmarks as potentially unstable. This snapshot is noisy and should not be used for close comparisons.
41+
> **Comparison warning:** Results from different machines or runs are not directly comparable. `pyperf` flagged 37 of 38 primitive benchmarks and 47 of 48 encoded benchmarks as potentially unstable. This snapshot is noisy and should not be used for close comparisons.
4242
43-
The snapshot used CPython 3.14.6 (64-bit) on macOS 15.5, arm64, with an Apple M2 CPU (8 cores: 4 performance and 4 efficiency). Garbage collection was enabled. The primitive suite ran at `2026-09-06T23:52:07.196571+00:00` from revision `c1f0a332e358bd358330471aab22edad48e7b5d6`; the encoded suite ran at `2026-09-07T03:03:35.663136+00:00` from revision `5ccf7af2957171180b5dc7aedeea5bb9a95d925e`.
43+
The snapshot used CPython 3.14.6 (64-bit) on macOS 15.5, arm64, with an Apple M2 CPU (8 cores: 4 performance and 4 efficiency). Garbage collection was enabled. The primitive suite ran at `2026-09-13T19:13:09.778895+00:00`; the encoded suite ran at `2026-09-13T19:45:06.276547+00:00`. Both suites ran from revision `354bbe98fd7fb4a4c22d21509a430bb1931ef752`.
4444

4545
## Install and validate
4646

‎results/raw/2026-09-13-cpython-3.14-encoded.json‎

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.

‎results/raw/2026-09-13-cpython-3.14-primitive.json‎

Lines changed: 1 addition & 0 deletions
Large diffs are not rendered by default.

‎results/reports/2026-09-13-cpython-3.14-encoded.html‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8">
5+
<meta name="viewport" content="width=device-width, initial-scale=1">
6+
<title>Serialization benchmark results</title>
7+
<style>:root { color-scheme: light dark; font-family: system-ui, sans-serif; }
8+
body { margin: 2rem; }
9+
main { margin: auto; max-width: 72rem; }
10+
nav { display: flex; gap: 1rem; margin-block: 1rem 2rem; }
11+
.warning { border-left: 0.3rem solid #c47f00; margin: 1.5rem 0; padding: 0.25rem 1rem; }
12+
.warning h2 { margin-bottom: 0.25rem; }
13+
.warning p { margin-top: 0; }
14+
dl { display: grid; gap: 0.25rem 1rem; grid-template-columns: max-content 1fr; }
15+
dt { font-weight: 700; }
16+
dd { margin: 0; }
17+
section { margin-block: 2rem; }
18+
table { border-collapse: collapse; width: 100%; }
19+
th, td { border: 1px solid #888; padding: 0.5rem; text-align: left; }
20+
th { font-weight: 700; }
21+
</style>
22+
</head>
23+
<body>
24+
<main>
25+
<h1>Serialization benchmark results</h1>
26+
<nav aria-label="Benchmark reports"><a href="2026-09-13-cpython-3.14-primitive.html">Primitive results</a> <a href="2026-09-13-cpython-3.14-encoded.html">Encoded results</a></nav>
27+
<aside class="warning" role="note"><h2>Comparison warning</h2><p>Results from different machines or runs are not directly comparable. pyperf flagged 47 of 48 benchmarks as potentially unstable. This noisy run should not be used for close comparisons.</p></aside>
28+
<section><h2>Environment</h2><dl><dt>Python</dt><dd>3.14.6 (64-bit) (cpython)</dd>
29+
<dt>Platform</dt><dd>macOS-15.5-arm64-arm-64bit-Mach-O</dd>
30+
<dt>CPU model</dt><dd>Unknown</dd>
31+
<dt>CPU count</dt><dd>8</dd>
32+
<dt>Git revision</dt><dd>354bbe98fd7fb4a4c22d21509a430bb1931ef752</dd>
33+
<dt>Benchmark version</dt><dd>0.1.0</dd>
34+
<dt>GC policy</dt><dd>enabled</dd>
35+
<dt>Run UTC</dt><dd>2026-09-13T19:45:06.276547+00:00</dd>
36+
<dt>Unit</dt><dd>second</dd></dl></section>
37+
<section><h2>Encoded / AVRO / decode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>Avro</td><td>1.12.2</td><td>9.34133 ms</td><td>4.26218 ms</td><td>107.051</td><td>120</td><td></td><td>19413 B</td></tr></tbody></table></section>
38+
<section><h2>Encoded / AVRO / decode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>Avro</td><td>1.12.2</td><td>110.494 µs</td><td>45.0209 µs</td><td>9050.25</td><td>120</td><td></td><td>153 B</td></tr></tbody></table></section>
39+
<section><h2>Encoded / AVRO / encode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>Avro</td><td>1.12.2</td><td>14.6538 ms</td><td>4.66952 ms</td><td>68.2415</td><td>120</td><td></td><td>19413 B</td></tr></tbody></table></section>
40+
<section><h2>Encoded / AVRO / encode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>Avro</td><td>1.12.2</td><td>183.359 µs</td><td>41.3883 µs</td><td>5453.78</td><td>120</td><td></td><td>153 B</td></tr></tbody></table></section>
41+
<section><h2>Encoded / JSON / decode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th><th scope="col">Speed vs baseline (higher is faster)</th></tr></thead><tbody><tr><td>mashumaro JSON</td><td>3.22</td><td>1.69806 ms</td><td>329.48 µs</td><td>588.907</td><td>120</td><td></td><td>62890 B</td><td>0.77x</td></tr>
42+
<tr><td>msgspec JSON</td><td>0.21.1</td><td>795.796 µs</td><td>470.457 µs</td><td>1256.6</td><td>120</td><td></td><td>53091 B</td><td>1.63x</td></tr>
43+
<tr><td>orjson</td><td>3.12.0</td><td>1.13317 ms</td><td>41.8863 µs</td><td>882.481</td><td>120</td><td></td><td>53091 B</td><td>1.15x</td></tr>
44+
<tr><td>Pydantic v2 JSON</td><td>2.13.5</td><td>1.3676 ms</td><td>295.037 µs</td><td>731.208</td><td>120</td><td></td><td>53091 B</td><td>0.95x</td></tr>
45+
<tr><td>serpyco-rs JSON</td><td>1.22.0</td><td>506.005 µs</td><td>184.733 µs</td><td>1976.27</td><td>120</td><td></td><td>53091 B</td><td>2.57x</td></tr>
46+
<tr><td>stdlib json</td><td>3.14.6</td><td>1.29906 ms</td><td>51.655 µs</td><td>769.787</td><td>120</td><td></td><td>53091 B</td><td>1.00x</td></tr></tbody></table></section>
47+
<section><h2>Encoded / JSON / decode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th><th scope="col">Speed vs baseline (higher is faster)</th></tr></thead><tbody><tr><td>mashumaro JSON</td><td>3.22</td><td>13.7959 µs</td><td>1.89666 µs</td><td>72485.3</td><td>120</td><td></td><td>584 B</td><td>1.06x</td></tr>
48+
<tr><td>msgspec JSON</td><td>0.21.1</td><td>3.49991 µs</td><td>466.973 ns</td><td>285721</td><td>120</td><td></td><td>487 B</td><td>4.18x</td></tr>
49+
<tr><td>orjson</td><td>3.12.0</td><td>8.06363 µs</td><td>795.214 ns</td><td>124014</td><td>120</td><td></td><td>487 B</td><td>1.82x</td></tr>
50+
<tr><td>Pydantic v2 JSON</td><td>2.13.5</td><td>8.20334 µs</td><td>867.558 ns</td><td>121902</td><td>120</td><td></td><td>487 B</td><td>1.78x</td></tr>
51+
<tr><td>serpyco-rs JSON</td><td>1.22.0</td><td>5.80227 µs</td><td>2.07368 µs</td><td>172346</td><td>120</td><td></td><td>487 B</td><td>2.52x</td></tr>
52+
<tr><td>stdlib json</td><td>3.14.6</td><td>14.638 µs</td><td>7.98278 µs</td><td>68315.3</td><td>120</td><td></td><td>487 B</td><td>1.00x</td></tr></tbody></table></section>
53+
<section><h2>Encoded / JSON / encode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th><th scope="col">Speed vs baseline (higher is faster)</th></tr></thead><tbody><tr><td>mashumaro JSON</td><td>3.22</td><td>977.036 µs</td><td>129.412 µs</td><td>1023.5</td><td>120</td><td></td><td>62890 B</td><td>1.58x</td></tr>
54+
<tr><td>msgspec JSON</td><td>0.21.1</td><td>170.965 µs</td><td>47.4382 µs</td><td>5849.16</td><td>120</td><td></td><td>53091 B</td><td>9.02x</td></tr>
55+
<tr><td>orjson</td><td>3.12.0</td><td>94.3509 µs</td><td>3.23088 µs</td><td>10598.7</td><td>120</td><td></td><td>53091 B</td><td>16.35x</td></tr>
56+
<tr><td>Pydantic v2 JSON</td><td>2.13.5</td><td>357.611 µs</td><td>27.5604 µs</td><td>2796.33</td><td>120</td><td></td><td>53091 B</td><td>4.31x</td></tr>
57+
<tr><td>serpyco-rs JSON</td><td>1.22.0</td><td>228.294 µs</td><td>169.872 µs</td><td>4380.32</td><td>120</td><td></td><td>53091 B</td><td>6.76x</td></tr>
58+
<tr><td>stdlib json</td><td>3.14.6</td><td>1.54295 ms</td><td>1.79893 ms</td><td>648.107</td><td>120</td><td></td><td>53091 B</td><td>1.00x</td></tr></tbody></table></section>
59+
<section><h2>Encoded / JSON / encode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th><th scope="col">Speed vs baseline (higher is faster)</th></tr></thead><tbody><tr><td>mashumaro JSON</td><td>3.22</td><td>12.4858 µs</td><td>2.37453 µs</td><td>80091.1</td><td>120</td><td></td><td>584 B</td><td>0.68x</td></tr>
60+
<tr><td>msgspec JSON</td><td>0.21.1</td><td>1.51777 µs</td><td>122.501 ns</td><td>658861</td><td>120</td><td></td><td>487 B</td><td>5.62x</td></tr>
61+
<tr><td>orjson</td><td>3.12.0</td><td>930.054 ns</td><td>77.5022 ns</td><td>1.07521e+06</td><td>120</td><td></td><td>487 B</td><td>9.17x</td></tr>
62+
<tr><td>Pydantic v2 JSON</td><td>2.13.5</td><td>6.0348 µs</td><td>2.04111 µs</td><td>165706</td><td>120</td><td></td><td>487 B</td><td>1.41x</td></tr>
63+
<tr><td>serpyco-rs JSON</td><td>1.22.0</td><td>3.08171 µs</td><td>1.61473 µs</td><td>324495</td><td>120</td><td></td><td>487 B</td><td>2.77x</td></tr>
64+
<tr><td>stdlib json</td><td>3.14.6</td><td>8.53035 µs</td><td>1.32526 µs</td><td>117228</td><td>120</td><td></td><td>487 B</td><td>1.00x</td></tr></tbody></table></section>
65+
<section><h2>Encoded / MESSAGEPACK / decode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>mashumaro MessagePack</td><td>3.22</td><td>1.00587 ms</td><td>331.554 µs</td><td>994.162</td><td>120</td><td></td><td>34653 B</td></tr>
66+
<tr><td>msgspec MessagePack</td><td>0.21.1</td><td>319.171 µs</td><td>267.792 µs</td><td>3133.11</td><td>120</td><td></td><td>34653 B</td></tr>
67+
<tr><td>ormsgpack</td><td>1.12.2</td><td>1.49505 ms</td><td>408.054 µs</td><td>668.874</td><td>120</td><td></td><td>34653 B</td></tr>
68+
<tr><td>serpyco-rs MessagePack</td><td>1.22.0</td><td>240.071 µs</td><td>68.6764 µs</td><td>4165.43</td><td>120</td><td></td><td>34653 B</td></tr></tbody></table></section>
69+
<section><h2>Encoded / MESSAGEPACK / decode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>mashumaro MessagePack</td><td>3.22</td><td>6.47899 µs</td><td>159.289 ns</td><td>154345</td><td>120</td><td></td><td>274 B</td></tr>
70+
<tr><td>msgspec MessagePack</td><td>0.21.1</td><td>2.5739 µs</td><td>258.339 ns</td><td>388516</td><td>120</td><td></td><td>274 B</td></tr>
71+
<tr><td>ormsgpack</td><td>1.12.2</td><td>12.8344 µs</td><td>3.08842 µs</td><td>77915.4</td><td>120</td><td></td><td>274 B</td></tr>
72+
<tr><td>serpyco-rs MessagePack</td><td>1.22.0</td><td>2.10466 µs</td><td>205.821 ns</td><td>475136</td><td>120</td><td></td><td>274 B</td></tr></tbody></table></section>
73+
<section><h2>Encoded / MESSAGEPACK / encode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>mashumaro MessagePack</td><td>3.22</td><td>477.639 µs</td><td>75.9269 µs</td><td>2093.63</td><td>120</td><td></td><td>34653 B</td></tr>
74+
<tr><td>msgspec MessagePack</td><td>0.21.1</td><td>95.1839 µs</td><td>2.53187 µs</td><td>10506</td><td>120</td><td></td><td>34653 B</td></tr>
75+
<tr><td>ormsgpack</td><td>1.12.2</td><td>228.224 µs</td><td>48.8295 µs</td><td>4381.66</td><td>120</td><td></td><td>34653 B</td></tr>
76+
<tr><td>serpyco-rs MessagePack</td><td>1.22.0</td><td>98.9967 µs</td><td>30.2495 µs</td><td>10101.3</td><td>120</td><td></td><td>34653 B</td></tr></tbody></table></section>
77+
<section><h2>Encoded / MESSAGEPACK / encode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>mashumaro MessagePack</td><td>3.22</td><td>4.75235 µs</td><td>462.914 ns</td><td>210422</td><td>120</td><td></td><td>274 B</td></tr>
78+
<tr><td>msgspec MessagePack</td><td>0.21.1</td><td>1.84773 µs</td><td>232.171 ns</td><td>541206</td><td>120</td><td></td><td>274 B</td></tr>
79+
<tr><td>ormsgpack</td><td>1.12.2</td><td>3.63077 µs</td><td>1.55775 µs</td><td>275423</td><td>120</td><td></td><td>274 B</td></tr>
80+
<tr><td>serpyco-rs MessagePack</td><td>1.22.0</td><td>1.09201 µs</td><td>121.661 ns</td><td>915740</td><td>120</td><td></td><td>274 B</td></tr></tbody></table></section>
81+
<section><h2>Encoded / PICKLE / decode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>pickle</td><td>3.14.6</td><td>777.633 µs</td><td>187.773 µs</td><td>1285.95</td><td>120</td><td></td><td>46842 B</td></tr></tbody></table></section>
82+
<section><h2>Encoded / PICKLE / decode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>pickle</td><td>3.14.6</td><td>5.59733 µs</td><td>512.613 ns</td><td>178657</td><td>120</td><td></td><td>529 B</td></tr></tbody></table></section>
83+
<section><h2>Encoded / PICKLE / encode_many / batch 100 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>pickle</td><td>3.14.6</td><td>603.517 µs</td><td>45.4374 µs</td><td>1656.95</td><td>120</td><td></td><td>46842 B</td></tr></tbody></table></section>
84+
<section><h2>Encoded / PICKLE / encode_one / batch 1 / dataclass</h2><table><thead><tr><th scope="col">Adapter</th><th scope="col">Version</th><th scope="col">Mean</th><th scope="col">Std dev</th><th scope="col">Operations/s</th><th scope="col">Samples</th><th scope="col">Prerelease</th><th scope="col">Payload</th></tr></thead><tbody><tr><td>pickle</td><td>3.14.6</td><td>7.83505 µs</td><td>2.02423 µs</td><td>127632</td><td>120</td><td></td><td>529 B</td></tr></tbody></table></section>
85+
</main>
86+
</body>
87+
</html>

0 commit comments

Comments
 (0)