Skip to content

Use octet_length for Hyper, Umbra, CedarDB - #1376

Merged
alexey-milovidov merged 8 commits into
ClickHouse:mainfrom
vogelsgesang:avogelsgesang-octet-length
Aug 15, 2026
Merged

Use octet_length for Hyper, Umbra, CedarDB#1376
alexey-milovidov merged 8 commits into
ClickHouse:mainfrom
vogelsgesang:avogelsgesang-octet-length

Conversation

@vogelsgesang

Copy link
Copy Markdown
Contributor

Switch Hyper, Umbra, and CedarDB to octet_length so Q27 and Q28 measure bytes, matching the intended ClickBench semantics while avoiding UTF-8 character scans when byte lengths are available from string offsets.

The queries for ClickHouse use length(URL) / length(Referer). In ClickHouse, length is byte-oriented (https://clickhouse.com/docs/sql-reference/functions/string-functions#length), in contrast to lengthUTF8. Given ClickHouse itself uses a byte-length, the intent of ClickBench was probably to measure bytes, not UTF8. Hyper / Umbra / CedarDB were accidentally using UTF8 lengths, since their length function aligns with Postgres and nobody noticed the difference.

Also see #981, #985

Switch Hyper, Umbra, and CedarDB to octet_length so Q27 and Q28 measure bytes, matching the intended ClickBench semantics while avoiding UTF-8 character scans when byte lengths are available from string offsets.

Rationale: apache/datafusion#23086
Prior DataFusion change: ClickHouse#985
@vogelsgesang

Copy link
Copy Markdown
Contributor Author

CC @toschmidt re the changes to Umbra
CC @lukasvogel re the changes to CedarDB

Please let know if you disagree with those changes. Happy to back them out of this PR, if you don't want them

Apply the same Q27 and Q28 byte-length semantics to the CedarDB and Hyper Parquet configurations so alternative configurations remain comparable with their main variants.
@vogelsgesang
vogelsgesang deployed to benchmark-approval August 14, 2026 21:00 — with GitHub Actions Active
@alexey-milovidov alexey-milovidov self-assigned this Aug 14, 2026
…uet-partitioned, hyper-parquet-single, umbra (c6a.4xlarge)
@github-actions

Copy link
Copy Markdown
Contributor

Results for cedardb are ready for: c6a.4xlarge.
Results for cedardb-parquet are ready for: c6a.4xlarge.
Results for hyper are ready for: c6a.4xlarge.
Results for hyper-parquet-partitioned are ready for: c6a.4xlarge.
Results for hyper-parquet-single are ready for: c6a.4xlarge.
Results for umbra are ready for: c6a.4xlarge.
The result files are committed as c355792.

Logs:

@alexey-milovidov alexey-milovidov added the machine:all PR benchmark on every machine type label Aug 15, 2026
@alexey-milovidov
alexey-milovidov deployed to benchmark-approval August 15, 2026 00:37 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown
Contributor

Results for cedardb are ready for: c6a.metal.
The result files are committed as a6a1000.
The run of cedardb on t3a.small did not produce results.
The run of hyper on c6a.large did not produce results.
The run of hyper on c8g.4xlarge did not produce results.
The run of hyper on c8g.metal-48xl did not produce results.
The run of hyper on t3a.small did not produce results.
The run of hyper-parquet-partitioned on c8g.4xlarge did not produce results.
The run of hyper-parquet-partitioned on c8g.metal-48xl did not produce results.
The run of hyper-parquet-single on c8g.4xlarge did not produce results.
The run of hyper-parquet-single on c8g.metal-48xl did not produce results.

Logs:

…uet-partitioned, hyper-parquet-single, umbra (c6a.2xlarge, c6a.4xlarge, c6a.large, c6a.metal, c6a.xlarge, c7a.metal-48xl, c8g.4xlarge, c8g.metal-48xl, t3a.small)
@github-actions

Copy link
Copy Markdown
Contributor

Results for cedardb are ready for: c6a.4xlarge, c6a.xlarge, c7a.metal-48xl, c8g.4xlarge, c8g.metal-48xl.
Results for cedardb-parquet are ready for: c6a.2xlarge, c6a.4xlarge, c6a.large, c6a.metal, c6a.xlarge, c7a.metal-48xl, c8g.4xlarge, c8g.metal-48xl, t3a.small.
Results for hyper are ready for: c6a.2xlarge, c6a.4xlarge, c6a.metal, c6a.xlarge, c7a.metal-48xl.
Results for hyper-parquet-partitioned are ready for: c6a.2xlarge, c6a.4xlarge, c6a.large, c6a.metal, c6a.xlarge, c7a.metal-48xl.
Results for hyper-parquet-single are ready for: c6a.2xlarge, c6a.4xlarge, c6a.large, c6a.metal, c6a.xlarge, c7a.metal-48xl.
Results for umbra are ready for: c6a.2xlarge, c6a.4xlarge, c6a.large, c6a.metal, c6a.xlarge, c7a.metal-48xl, c8g.4xlarge, c8g.metal-48xl, t3a.small.
The result files are committed as 46596a6.

Logs:

@github-actions

Copy link
Copy Markdown
Contributor

Results for hyper-parquet-partitioned are ready for: t3a.small.
Results for hyper-parquet-single are ready for: t3a.small.
The result files are committed as 026bf5a.

Logs:

@github-actions

Copy link
Copy Markdown
Contributor

Results for cedardb are ready for: c6a.2xlarge.
The result files are committed as ab0e1d5.

Logs:

@github-actions

Copy link
Copy Markdown
Contributor

The run of cedardb on c6a.large did not produce results.

Logs:

@alexey-milovidov
alexey-milovidov merged commit c68a9ac into ClickHouse:main Aug 15, 2026
1 check passed
@vogelsgesang

Copy link
Copy Markdown
Contributor Author

just for reference (and so I can find it again later):

c6a.4xlarge, median of the two warm runs:

Database Q27 Q28 Aggregate effect
Hyper 458 → 146 ms (3.14×) 7.109 → 6.912 s (2.8%) ~2.7%
Umbra 290 → 115 ms (2.53×) 1.526 → 1.397 s (8.5%) ~2.3%
CedarDB 275 → 108 ms (2.54×) 3.198 → 3.097 s (3.2%) ~2.2%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

machine:all PR benchmark on every machine type

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants