Build benchmarks with whole-program optimization

[connortsui20]

Rationale for this change

profile.bench builds with codegen-units = 16 and no LTO. Under those settings, adding unrelated code to a crate reshuffles which functions share a codegen unit, and cross-unit inlining changes for functions whose source did not change. A benchmark moves without its code moving, which is indistinguishable from a real regression.

The effect is large enough to mislead a review. On #9255, which adds a row scalar function framework to vortex-array, CodSpeed reported 14-16% regressions across take_filter_list_* and list_sum_small. None of those have a source change on that branch, and every take_filter_primitive_* benchmark held. Cargo.lock is identical and nothing was added to the session registry, so codegen-unit partitioning is what is left.

Benchmarks exist to attribute a change in measurement to a change in code. codegen-units = 1 with fat LTO removes the variable.

What changes are included in this PR?

One profile, two settings: codegen-units = 16 to 1, and lto = false to true.

What APIs are changed? Are there any user-facing changes?

None. profile.bench affects cargo bench and cargo codspeed build --profile bench only.

Notes

profile.release deliberately keeps LTO off, with a comment recording why: Vortex's performance must not depend on a downstream crate enabling it. That argument is about shipped code. It does not extend to a profile whose only job is measuring a diff, and the two profiles already disagree on codegen-units.

The trade-off is benchmark build time. Fat LTO over one codegen unit is materially slower than 16 parallel units, and every CodSpeed shard pays it.

Open question: is the noise this removes worth the CI time it costs, or is the better fix to keep the current profile and treat sub-20% moves in untouched code as unattributable?

Merging this invalidates comparison against any baseline built with the old settings. The first CodSpeed run on this PR compares a fat-LTO head against a codegen-units = 16 base, so its numbers describe the profile change rather than any code change. Runs after it lands are the meaningful ones.

---

Generated by Claude Code

[codspeed-hq]

Merging this PR will improve performance by 29.51%

⚡ 115 improved benchmarks
✅ 146 untouched benchmarks
⏩ 1719 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	slice_vortex_buffer	14.2 µs	5.6 µs	×2.5
⚡	Simulation	from_iter_bit_buffer[65536]	233.3 µs	136.8 µs	+70.59%
⚡	Simulation	slice_tight_loop_arrow[65536]	94.6 µs	55.7 µs	+69.95%
⚡	Simulation	per_element_value[(16384, 0.01)]	181.7 µs	111.8 µs	+62.58%
⚡	Simulation	from_iter_bit_buffer[16384]	67.8 µs	41.9 µs	+61.82%
⚡	Simulation	value_arrow_buffer[65536]	895.1 µs	570.8 µs	+56.82%
⚡	Simulation	value_arrow_buffer[16384]	224.1 µs	142.9 µs	+56.8%
⚡	Simulation	value_vortex_buffer[65536]	895.1 µs	570.8 µs	+56.8%
⚡	Simulation	value_vortex_buffer[16384]	224.1 µs	143 µs	+56.73%
⚡	Simulation	value_arrow_buffer[2048]	28.4 µs	18.1 µs	+56.69%
⚡	Simulation	value_arrow_buffer[1024]	14.4 µs	9.2 µs	+56.56%
⚡	Simulation	set_slices[(16384, 0.01)]	23.4 µs	15 µs	+56.25%
⚡	Simulation	value_vortex_buffer[2048]	28.4 µs	18.2 µs	+56.11%
⚡	Simulation	per_element_value[(16384, 0.1)]	202.9 µs	130.1 µs	+55.93%
⚡	Simulation	value_vortex_buffer[1024]	14.4 µs	9.3 µs	+55.42%
⚡	Simulation	append_buffer_arrow_buffer[128]	7.6 µs	4.9 µs	+53.46%
⚡	Simulation	value_arrow_buffer[128]	2.2 µs	1.4 µs	+53.23%
⚡	Simulation	bitand_owned_lhs_vortex_buffer[128]	7.3 µs	4.8 µs	+51.88%
⚡	Simulation	from_bool_slice[1024]	5 µs	3.3 µs	+50.05%
⚡	Simulation	bitwise_and_vortex_buffer[1024]	6.2 µs	4.2 µs	+48.56%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/lto-bench (f5a4567) with develop (4e3be5b)}

Footnotes

1. 1719 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[connortsui20]

Closing. The profile change does not isolate what it was meant to isolate. Fat LTO over one codegen unit made a wide set of benchmarks faster, including many with no relationship to the change under test, so it substitutes one confound for a larger one. A CodSpeed run against this base still cannot attribute a move to the diff.

The benchmarks that got faster under LTO are worth a separate look. They point at cross-crate inlining that the default build leaves on the table, which is an optimization opportunity in shipped code rather than a benchmarking setting. That work is not tracked yet.

#9255 is back on develop with profile.bench unchanged. The branch stays if this is worth revisiting.

---

Generated by Claude Code