Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/npm-build-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ jobs:
# silently on the v0.2.2 release. Exact-string match avoids
# the whole class of regex-escaping bugs.
REQUIRED_JSON='[
"rustfmt","clippy","cargo-audit","cargo-deny",
"rustfmt","clippy (ubuntu-latest)","clippy (windows-latest)",
"cargo-audit","cargo-deny",
"version consistency","publish dry-run",
"test (ubuntu-latest)","test (macos-14)","test (windows-latest)",
"hyperdb-api-node (build + smoke)"
Expand Down
24 changes: 22 additions & 2 deletions docs/BENCHMARK_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,13 @@ Contributions welcome for additional platforms — paste the
summary table under the appropriate section and include the host
block from the suite's stdout.

> **Units.** `MB/sec` means **decimal** megabytes per second — 10^6 bytes/s —
> matching what `benches/common.rs` emits and the conventional unit for I/O
> throughput. Every other unit the harness prints (`fmt_count`, `fmt_rate`,
> `fmt_size`) is decimal too. The one exception in this document is flagged
> inline: the [native Windows tables](#platform-windows-x86_64-native) predate
> the harness fix and are still in MiB/s.

### Platform: macOS (Apple Silicon)

**Hardware / software**
Expand Down Expand Up @@ -332,7 +339,20 @@ unchanged — opt into `ArrowInserter` and `executeQueryColumnar` /

#### Rust suite — 100M rows per workload, 4 parallel workers, TCP loopback

| Workload | Variant | Flavor | Rows | Time (s) | Rows/sec | MB/sec |
> **The `MB/sec` column below is MiB/s (2^20 B/s), not decimal MB/s.** These
> numbers were captured on 2026-05-02, when the harness divided byte counts by
> 1024² while labelling the result `MB`. That has since been corrected to
> decimal, so this table reads **4.86% low** against every other table here.
>
> The cells are left exactly as measured rather than multiplied by 1.048576.
> The run is also stale on three other axes — `hyperdb-api` 0.1.0-rc.1, rustc
> 1.92.0, and an unrecorded `hyperd` predating the current `0.0.26479` pin — so
> it needs re-measuring regardless, and an arithmetic conversion would make
> stale data look freshly sampled. **These figures will be restated from a real
> run the next time the suite is executed on native Windows**; until then,
> compare them only against each other, never against the macOS tables.

| Workload | Variant | Flavor | Rows | Time (s) | Rows/sec | MiB/sec |
|---|---|---|---:|---:|---:|---:|
| insert.bulk | AsyncArrowInserter | async | 100.00M | 18.563 | 5.39 M/s | 123.3 |
| insert.bulk | AsyncArrowInserter × 4 | async | 100.00M | 4.931 | 20.28 M/s | 464.1 |
Expand All @@ -351,7 +371,7 @@ unchanged — opt into `ArrowInserter` and `executeQueryColumnar` /

**Headline takeaways (Rust, native Windows / i9-10980XE):**

- **Parallel async inserts** are the throughput-dominant path — `spawn_blocking + ChunkSender × 4` reaches **20.9 M rows/s / 479 MB/s**, ~2× faster than sync inserts and within ~30% of the TCP loopback ceiling on this box. The 4-way parallel insert numbers are roughly on par with macOS / M3 Max in absolute throughput, suggesting hyperd's ingest path is *not* the bottleneck here.
- **Parallel async inserts** are the throughput-dominant path — `spawn_blocking + ChunkSender × 4` reaches **20.9 M rows/s / 479 MiB/s**, ~2× faster than sync inserts and within ~30% of the TCP loopback ceiling on this box. The 4-way parallel insert numbers are roughly on par with macOS / M3 Max in absolute throughput, suggesting hyperd's ingest path is *not* the bottleneck here.
- **Single-connection sync query** went from 2.89 M/s (pre-2026-05 tuning) to **7.08 M/s** — a 2.5× improvement — after the read-window + TCP-buffer changes documented below.
- **Single-connection sync inserts on Windows lag** native Linux/macOS by ~5× even after tuning. This is a residual `hyperd`-side gap; the parallel paths hide it because they exercise multiple ingest threads.

Expand Down
10 changes: 10 additions & 0 deletions hyperdb-api/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
since the rename to `hyperdb-bootstrap`. The command failed with
"package(s) `hyperd-bootstrap` not found in workspace", which is the first
thing a new user saw when `HYPERD_PATH` was unset.
- The benchmark examples reported **MiB/s under an `MB/s` label**. `fmt_mb`,
`BenchRecord::mb_per_sec` and the `memory_*_mb` helpers in
`benches/common.rs`, plus the equivalent divisions in `benchmark`,
`arrow_batching_benchmark` and `async_parallel_benchmark`, all divided byte
counts by 1024². Their sibling formatters (`fmt_count`, `fmt_rate`,
`fmt_size`) were already decimal, so a single `MB/sec` column in
[docs/BENCHMARK_GUIDE.md](../docs/BENCHMARK_GUIDE.md) carried two different
units depending on which platform produced the row — a 4.86% discrepancy.
All `MB`-labelled output is now decimal (10^6). Installed-RAM reporting stays
binary, since RAM is conventionally quoted that way.

### Changed

Expand Down
17 changes: 11 additions & 6 deletions hyperdb-api/benches/arrow_batching_benchmark.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@ use hyperdb_api::{
HyperProcess, Parameters, Result, SqlType, TableDefinition, TransportMode,
};

/// Bytes in one megabyte — decimal (10^6), matching the rest of the bench
/// suite. This benchmark is standalone (it does not pull in `common.rs`), so
/// it carries its own copy of the constant.
const BYTES_PER_MB: f64 = 1_000_000.0;

const DEFAULT_ROW_COUNT: usize = 10_000_000;
const BATCH_SIZE: usize = 100_000;

Expand Down Expand Up @@ -158,7 +163,7 @@ fn main() -> Result<()> {

// Print database file size
if let Ok(metadata) = std::fs::metadata(async_db_path) {
let size_mb = metadata.len() as f64 / (1024.0 * 1024.0);
let size_mb = metadata.len() as f64 / BYTES_PER_MB;
println!("\nDatabase file size: {size_mb:.2} MB");
}

Expand Down Expand Up @@ -237,10 +242,10 @@ fn main() -> Result<()> {

let tcp_rows_per_sec = tcp_result.rows as f64 / tcp_result.elapsed.as_secs_f64();
let tcp_mb_per_sec =
tcp_result.total_bytes as f64 / (1024.0 * 1024.0) / tcp_result.elapsed.as_secs_f64();
tcp_result.total_bytes as f64 / BYTES_PER_MB / tcp_result.elapsed.as_secs_f64();
let ipc_rows_per_sec = ipc_result.rows as f64 / ipc_result.elapsed.as_secs_f64();
let ipc_mb_per_sec =
ipc_result.total_bytes as f64 / (1024.0 * 1024.0) / ipc_result.elapsed.as_secs_f64();
ipc_result.total_bytes as f64 / BYTES_PER_MB / ipc_result.elapsed.as_secs_f64();
let speedup = tcp_result.elapsed.as_secs_f64() / ipc_result.elapsed.as_secs_f64();

println!(
Expand Down Expand Up @@ -386,7 +391,7 @@ fn run_benchmark(
format_number(rows as usize),
elapsed.as_secs_f64(),
rows as f64 / elapsed.as_secs_f64(),
total_bytes as f64 / (1024.0 * 1024.0) / elapsed.as_secs_f64()
total_bytes as f64 / BYTES_PER_MB / elapsed.as_secs_f64()
);

Ok(BenchmarkResult {
Expand All @@ -398,7 +403,7 @@ fn run_benchmark(

fn print_result_row_wide(name: &str, result: &BenchmarkResult, speedup: f64) {
let rows_per_sec = result.rows as f64 / result.elapsed.as_secs_f64();
let mb_per_sec = result.total_bytes as f64 / (1024.0 * 1024.0) / result.elapsed.as_secs_f64();
let mb_per_sec = result.total_bytes as f64 / BYTES_PER_MB / result.elapsed.as_secs_f64();

println!(
"║ {:29} │ {:8.2} │ {:11} │ {:6.1} │ {:8.2}x ║",
Expand Down Expand Up @@ -566,7 +571,7 @@ async fn run_async_benchmark(
format_number(rows as usize),
elapsed.as_secs_f64(),
rows as f64 / elapsed.as_secs_f64(),
total_bytes as f64 / (1024.0 * 1024.0) / elapsed.as_secs_f64()
total_bytes as f64 / BYTES_PER_MB / elapsed.as_secs_f64()
);

Ok(BenchmarkResult {
Expand Down
10 changes: 5 additions & 5 deletions hyperdb-api/benches/async_parallel_benchmark.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ use hyperdb_api::{
InsertChunk, Parameters, Result, SqlType, TableDefinition,
};

use common::{BYTES_PER_ROW, fmt_count, fmt_rate};
use common::{BYTES_PER_MB, BYTES_PER_ROW, fmt_count, fmt_rate};

/// Await all `handles`, converting join errors into `hyperdb_api::Error`
/// and collecting successful results. Replaces `futures::try_join_all`
Expand Down Expand Up @@ -236,7 +236,7 @@ impl BenchTotals {
self.total_rows() as f64 / self.wall_secs
}
fn agg_mb_per_sec(&self) -> f64 {
(self.total_bytes() as f64) / (1024.0 * 1024.0) / self.wall_secs
(self.total_bytes() as f64) / BYTES_PER_MB / self.wall_secs
}
/// Ratio of summed per-worker time to wall-clock time. ~N means
/// near-perfect parallelism; 1.0 means fully serial.
Expand Down Expand Up @@ -403,7 +403,7 @@ async fn arrow_worker(
worker_id,
fmt_count(rows),
worker_time.as_secs_f64(),
(total_bytes as f64) / (1024.0 * 1024.0) / worker_time.as_secs_f64()
(total_bytes as f64) / BYTES_PER_MB / worker_time.as_secs_f64()
);

Ok(WorkerResult {
Expand Down Expand Up @@ -531,7 +531,7 @@ fn chunk_sender_worker(
worker_id,
fmt_count(rows),
worker_time.as_secs_f64(),
(total_bytes as f64) / (1024.0 * 1024.0) / worker_time.as_secs_f64()
(total_bytes as f64) / BYTES_PER_MB / worker_time.as_secs_f64()
);

Ok(WorkerResult {
Expand Down Expand Up @@ -674,7 +674,7 @@ async fn query_worker(
worker_id,
fmt_count(rows),
worker_time.as_secs_f64(),
(bytes as f64) / (1024.0 * 1024.0) / worker_time.as_secs_f64().max(1e-9),
(bytes as f64) / BYTES_PER_MB / worker_time.as_secs_f64().max(1e-9),
checksum
);

Expand Down
8 changes: 4 additions & 4 deletions hyperdb-api/benches/benchmark.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ use std::sync::Arc;
use std::sync::mpsc;
use std::thread;

use common::{ResourceMonitor, ResourceStats, SAMPLE_INTERVAL_MS};
use common::{BYTES_PER_MB, ResourceMonitor, ResourceStats, SAMPLE_INTERVAL_MS};

// Default 10M rows for comparison with C++ benchmark
const DEFAULT_ROW_COUNT: i64 = 10_000_000;
Expand Down Expand Up @@ -98,12 +98,12 @@ fn bytes_per_row() -> usize {
24
}

/// Calculates MB/sec from bytes and elapsed time.
/// Calculates decimal MB/sec from bytes and elapsed time.
fn mb_per_sec(bytes: f64, elapsed_secs: f64) -> f64 {
if elapsed_secs <= 0.0 {
return 0.0;
}
bytes / elapsed_secs / (1024.0 * 1024.0)
bytes / elapsed_secs / BYTES_PER_MB
}

/// Result of a benchmark run including timing and resource stats.
Expand Down Expand Up @@ -1223,7 +1223,7 @@ fn main() -> Result<()> {
// Print database file size before deletion
if let Ok(metadata) = std::fs::metadata(db_path) {
let size_bytes = metadata.len();
let size_mb = size_bytes as f64 / (1024.0 * 1024.0);
let size_mb = size_bytes as f64 / BYTES_PER_MB;
println!("\nDatabase file size: {size_mb:.2} MB ({size_bytes} bytes)");
}

Expand Down
34 changes: 26 additions & 8 deletions hyperdb-api/benches/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
//! - one `ResourceStats` + `ResourceMonitor` is the sole source of
//! CPU/memory metrics — no more copy-pastes drifting apart.
//! - formatting helpers (`fmt_count`, `fmt_rate`, `fmt_size`,
//! `fmt_mb`) produce identical output everywhere.
//! `fmt_mb`) produce identical output everywhere, all in decimal
//! units (see [`BYTES_PER_MB`]).
//! - `HOST_ENV` collects the OS, CPU, RAM, Rust version, hyperd git
//! hash so the unified result tables in `BENCHMARK_GUIDE.md` are
//! self-describing.
Expand Down Expand Up @@ -64,7 +65,9 @@ pub(crate) const BYTES_PER_ROW: usize = 24;
/// trivial (`SELECT COUNT(*)`, `SELECT SUM(value)` etc).
#[inline]
pub(crate) fn gen_id(start_id: i64, i: i64) -> i32 {
(start_id + i) as i32
// Narrowing `as` would silently wrap past 2^31 rows and emit duplicate,
// negative IDs — corrupting the very throughput numbers being measured.
i32::try_from(start_id + i).expect("benchmark row IDs must fit the `id INT` column")
}
#[inline]
pub(crate) fn gen_sensor_id(id: i32) -> i32 {
Expand Down Expand Up @@ -110,14 +113,14 @@ impl ResourceStats {
} else {
let avg =
self.memory_samples.iter().sum::<u64>() as f64 / self.memory_samples.len() as f64;
avg / (1024.0 * 1024.0)
avg / BYTES_PER_MB
}
}
pub(crate) fn memory_max_mb(&self) -> f64 {
self.memory_samples.iter().copied().max().unwrap_or(0) as f64 / (1024.0 * 1024.0)
self.memory_samples.iter().copied().max().unwrap_or(0) as f64 / BYTES_PER_MB
}
pub(crate) fn memory_min_mb(&self) -> f64 {
self.memory_samples.iter().copied().min().unwrap_or(0) as f64 / (1024.0 * 1024.0)
self.memory_samples.iter().copied().min().unwrap_or(0) as f64 / BYTES_PER_MB
}
}

Expand Down Expand Up @@ -177,6 +180,17 @@ impl ResourceMonitor {
// Formatting helpers.
// =============================================================================

/// Bytes in one megabyte.
///
/// **Decimal (10^6), not binary (2^20).** Every unit this module reports is
/// decimal: `fmt_count`, `fmt_rate` and `fmt_size` were already, but the
/// `MB`-labelled helpers divided by 1024² and so emitted MiB under an `MB`
/// label. That made a single `MB/sec` column in `BENCHMARK_GUIDE.md` carry two
/// different units depending on which platform's run produced the row — a
/// 4.86% discrepancy. Decimal is also the conventional unit for I/O
/// throughput. Anything labelled `MB` here divides by this constant.
pub(crate) const BYTES_PER_MB: f64 = 1_000_000.0;

/// Format a row count like `123.4K`, `12.3M`, `1.23B`.
pub(crate) fn fmt_count(n: u64) -> String {
if n >= 1_000_000_000 {
Expand All @@ -203,12 +217,12 @@ pub(crate) fn fmt_rate(rows_per_sec: f64) -> String {
}
}

/// Format MB/sec with two decimals.
/// Format decimal MB/sec with one decimal place.
pub(crate) fn fmt_mb(bytes: usize, elapsed_secs: f64) -> String {
if elapsed_secs <= 0.0 {
return "—".to_string();
}
let mb = bytes as f64 / (1024.0 * 1024.0);
let mb = bytes as f64 / BYTES_PER_MB;
format!("{:.1} MB/s", mb / elapsed_secs)
}

Expand Down Expand Up @@ -257,6 +271,9 @@ impl HostEnv {
.unwrap_or_default();
let physical = sysinfo::System::physical_core_count().unwrap_or(cpus.len());
let logical = cpus.len();
// Deliberately binary (2^30), unlike the decimal `MB` throughput
// units above: installed RAM is universally quoted in binary units,
// so a 36 GiB machine must read "36.0 GB" here and not "38.7".
let total_memory_gb = sys.total_memory() as f64 / (1024.0 * 1024.0 * 1024.0);

HostEnv {
Expand Down Expand Up @@ -358,11 +375,12 @@ impl BenchRecord {
self.rows as f64 / self.elapsed_secs
}
}
/// Decimal megabytes per second — see [`BYTES_PER_MB`].
pub(crate) fn mb_per_sec(&self) -> f64 {
if self.elapsed_secs <= 0.0 {
0.0
} else {
(self.bytes as f64 / (1024.0 * 1024.0)) / self.elapsed_secs
(self.bytes as f64 / BYTES_PER_MB) / self.elapsed_secs
}
}
}
Expand Down
Loading
Loading