[sinatra] Simplify total calculation for baseline11 by p8 · Pull Request #1306 · MDA2AV/HttpArena

p8 · 2026-08-24T19:17:55Z

This makes it consistent with other calls where we also sum the a and b params.

p8 · 2026-08-24T19:23:25Z

/benchmark -f sinatra

github-actions · 2026-08-24T19:23:46Z

👋 Benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-08-24T23:35:46Z

Benchmark Results

Framework: sinatra | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	92,503	4650.2%	6.6GiB	-5.8%	-1.5%
baseline	4096	61,083	3969.7%	4.9GiB	+4.4%	~0%
pipelined	512	473,671	6446.6%	2.0GiB	+10.1%	-4.8%
pipelined	4096	463,619	6399.3%	2.1GiB	+8.5%	~0%
limited-conn	512	38,124	2871.1%	4.0GiB	+2.3%	+14.3%
limited-conn	4096	40,614	3089.8%	4.3GiB	+1.9%	~0%
json	4096	209,087	6303.5%	4.9GiB	+3.7%	+6.5%
json-comp	512	115,368	6419.4%	4.6GiB	-2.3%	~0%
json-comp	4096	128,701	6406.2%	4.7GiB	+3.9%	+6.8%
json-comp	16384	126,179	6279.3%	5.0GiB	+3.9%	+11.1%
json-tls	4096	194,628	6487.8%	6.9GiB	+5.0%	+15.0%
upload	32	951	3056.2%	8.4GiB	+0.2%	-2.3%
upload	256	916	6515.0%	6.8GiB	-0.5%	-18.1%
api-4	256	12,278	400.5%	380MiB	-23.6%	-12.0%
api-16	1024	43,150	1621.1%	1.2GiB	+6.4%	-7.7%
static	1024	23,460	6510.6%	7.0GiB	-0.1%	+6.1%
static	4096	23,358	6472.7%	7.1GiB	+0.3%	+1.4%
static	6800	23,567	6482.7%	6.8GiB	+1.5%	-4.2%
static-tls	1024	21,740	6508.3%	7.0GiB	-0.2%	-2.8%
static-tls	4096	21,454	6507.4%	7.2GiB	-0.1%	-2.7%
static-tls	6800	21,168	6509.6%	7.4GiB	~0%	~0%
async-db	1024	106,269	6150.4%	3.2GiB	+5.7%	+14.3%
crud	4096	63,709	3006.7%	3.8GiB	-1.4%	-2.6%

Full log

  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   200.13ms   181.40ms   470.20ms   738.70ms   925.40ms

  50273 requests in 10.00s, 50273 responses
  Throughput: 5.03K req/s
  Bandwidth:  12.64MB/s
  Status codes: 2xx=30832, 3xx=0, 4xx=0, 5xx=19441
  Latency samples: 50273 / 50273 responses (100.0%)
  Latency overflow (>5s): 3
  Reconnects: 1557
  Per-template: 10165,10196,9897,10009,10006
  Per-template-ok: 6321,6201,6063,6200,6047

  WARNING: 19441/50273 responses (38.7%) had unexpected status (expected 2xx)
[info] CPU 835.9% | Mem 1.9GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.95ms   7.08ms   12.90ms   21.80ms   390.50ms

  1064111 requests in 10.00s, 1064113 responses
  Throughput: 106.39K req/s
  Bandwidth:  429.53MB/s
  Status codes: 2xx=1062698, 3xx=0, 4xx=0, 5xx=1415
  Latency samples: 1064113 / 1064113 responses (100.0%)
  Reconnects: 42293
  Per-template: 212267,213024,213555,212875,212392
  Per-template-ok: 211985,212772,213192,212609,212140

  WARNING: 1415/1064113 responses (0.1%) had unexpected status (expected 2xx)
[info] CPU 6150.4% | Mem 3.2GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.94ms   6.80ms   12.70ms   21.80ms   418.90ms

  1055503 requests in 10.00s, 1055504 responses
  Throughput: 105.53K req/s
  Bandwidth:  426.10MB/s
  Status codes: 2xx=1054017, 3xx=0, 4xx=0, 5xx=1487
  Latency samples: 1055504 / 1055504 responses (100.0%)
  Reconnects: 41963
  Per-template: 210541,211139,211677,211531,210615
  Per-template-ok: 210216,210882,211423,211165,210330

  WARNING: 1487/1055504 responses (0.1%) had unexpected status (expected 2xx)
[info] CPU 6256.0% | Mem 3.1GiB

=== Best: 106269 req/s (CPU: 6150.4%, Mem: 3.2GiB) ===
[info] input BW: 7.09MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/sinatra.json
[warn] sinatra async-db/1024: container log was 133435085 bytes, truncated to 8000000
httparena-bench-sinatra
httparena-bench-sinatra

==============================================
=== sinatra / crud / 4096c (tool=gcannon) ===
==============================================
[info] resetting postgres for a clean per-profile baseline
[info] starting postgres sidecar
httparena-postgres
[info] postgres ready (seeded)
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   66.01ms   65.80ms   74.70ms   91.60ms   288.80ms

  893964 requests in 15.00s, 889996 responses
  Throughput: 59.32K req/s
  Bandwidth:  18.67MB/s
  Status codes: 2xx=889996, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 889996 / 889996 responses (100.0%)
  Reconnects: 3524
  Per-template: 44971,44951,44482,44220,44314,44525,44524,44341,44119,44064,44440,44175,44539,44322,44184,44053,44166,46026,44928,44652
  Per-template-ok: 44971,44951,44482,44220,44314,44525,44524,44341,44119,44064,44440,44175,44539,44322,44184,44053,44166,46026,44928,44652
[info] CPU 3407.0% | Mem 3.7GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   61.47ms   63.50ms   70.10ms   81.00ms   280.50ms

  958093 requests in 15.00s, 955638 responses
  Throughput: 63.70K req/s
  Bandwidth:  19.80MB/s
  Status codes: 2xx=955638, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 955637 / 955638 responses (100.0%)
  Reconnects: 4077
  Per-template: 47325,48121,47755,47442,47561,48018,47831,47755,47413,47801,47964,47833,47473,47954,47587,47735,46775,49642,47934,47718
  Per-template-ok: 47325,48121,47755,47442,47561,48018,47831,47755,47413,47801,47964,47833,47473,47954,47587,47735,46775,49642,47934,47718
[info] CPU 3006.7% | Mem 3.8GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   63.20ms   64.80ms   71.50ms   83.60ms   98.00ms

  941577 requests in 15.00s, 937609 responses
  Throughput: 62.50K req/s
  Bandwidth:  19.42MB/s
  Status codes: 2xx=937609, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 937608 / 937609 responses (100.0%)
  Reconnects: 3997
  Per-template: 46697,47376,46821,46637,46573,47173,46846,46592,46769,46864,46834,46556,46876,46694,47097,46578,46016,48909,46682,47018
  Per-template-ok: 46697,47376,46821,46637,46573,47173,46846,46592,46769,46864,46834,46556,46876,46694,47097,46578,46016,48909,46682,47018
[info] CPU 3019.7% | Mem 3.9GiB

=== Best: 63709 req/s (CPU: 3006.7%, Mem: 3.8GiB) ===
[info] input BW: 5.47MB/s (avg template: 90 bytes)
[info] saved results/crud/4096/sinatra.json
httparena-bench-sinatra
httparena-bench-sinatra
[info] skip: sinatra does not subscribe to fortunes
[info] skip: sinatra does not subscribe to baseline-h2
[info] skip: sinatra does not subscribe to static-h2
[info] skip: sinatra does not subscribe to baseline-h2c
[info] skip: sinatra does not subscribe to json-h2c
[info] skip: sinatra does not subscribe to baseline-h3
[info] skip: sinatra does not subscribe to static-h3
[info] skip: sinatra does not subscribe to gateway-64
[info] skip: sinatra does not subscribe to gateway-h3
[info] skip: sinatra does not subscribe to production-stack
[info] skip: sinatra does not subscribe to unary-grpc
[info] skip: sinatra does not subscribe to unary-grpc-tls
[info] skip: sinatra does not subscribe to echo-ws
[info] skip: sinatra does not subscribe to echo-ws-pipeline
[info] skip: sinatra does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/sinatra.json - 23 new, 23 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

This makes it consistent with other calls where we also sum the a and b params.

MDA2AV · 2026-08-25T00:52:09Z

/benchmark -f sinatra --save

github-actions · 2026-08-25T00:52:18Z

👋 Benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-08-25T01:20:36Z

Benchmark Results

Framework: sinatra | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	90,597	4548.0%	6.3GiB	-7.8%	-6.0%
baseline	4096	59,276	3709.8%	4.8GiB	+1.3%	-2.0%
pipelined	512	483,705	6390.2%	2.0GiB	+12.4%	-4.8%
pipelined	4096	486,654	6485.8%	2.0GiB	+13.9%	-4.8%
limited-conn	512	37,267	2811.9%	3.8GiB	~0%	+8.6%
limited-conn	4096	40,307	3123.7%	4.3GiB	+1.2%	~0%
json	4096	208,194	6389.9%	4.8GiB	+3.3%	+4.3%
json-comp	512	117,317	6439.8%	4.9GiB	-0.7%	+6.5%
json-comp	4096	127,744	6446.9%	4.8GiB	+3.1%	+9.1%
json-comp	16384	125,368	6298.4%	5.0GiB	+3.2%	+11.1%
json-tls	4096	187,170	6532.8%	6.6GiB	+0.9%	+10.0%
upload	32	957	3079.6%	8.2GiB	+0.8%	-4.7%
upload	256	905	6348.1%	8.4GiB	-1.7%	+1.2%
api-4	256	12,359	400.3%	388MiB	-23.1%	-10.2%
api-16	1024	43,691	1629.4%	1.2GiB	+7.8%	-7.7%
static	1024	23,629	6508.1%	7.0GiB	+0.6%	+6.1%
static	4096	23,440	6506.1%	7.1GiB	+0.6%	+1.4%
static	6800	23,501	6467.5%	6.9GiB	+1.2%	-2.8%
static-tls	1024	21,699	6516.7%	7.2GiB	-0.3%	~0%
static-tls	4096	21,362	6452.4%	7.5GiB	-0.6%	+1.4%
static-tls	6800	21,013	6403.7%	7.8GiB	-0.8%	+5.4%
async-db	1024	108,825	6244.8%	3.2GiB	+8.3%	+14.3%
crud	4096	60,997	2894.9%	3.9GiB	-5.6%	~0%

Full log

[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   191.65ms   148.30ms   472.20ms   724.70ms   900.30ms

  52434 requests in 10.00s, 52434 responses
  Throughput: 5.24K req/s
  Bandwidth:  13.67MB/s
  Status codes: 2xx=33009, 3xx=0, 4xx=0, 5xx=19425
  Latency samples: 52434 / 52434 responses (100.0%)
  Reconnects: 1615
  Per-template: 10467,10509,10368,10469,10621
  Per-template-ok: 6660,6661,6377,6496,6815

  WARNING: 19425/52434 responses (37.0%) had unexpected status (expected 2xx)
[info] CPU 850.2% | Mem 1.9GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.81ms   8.23ms   14.10ms   21.30ms   29.40ms

  1088249 requests in 10.00s, 1088252 responses
  Throughput: 108.80K req/s
  Bandwidth:  439.93MB/s
  Status codes: 2xx=1088252, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1088249 / 1088252 responses (100.0%)
  Reconnects: 43265
  Per-template: 217124,217690,218102,218130,217203
  Per-template-ok: 217124,217690,218102,218130,217203
[info] CPU 6244.8% | Mem 3.2GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   8.91ms   8.37ms   14.40ms   20.90ms   28.60ms

  1071456 requests in 10.00s, 1071456 responses
  Throughput: 107.12K req/s
  Bandwidth:  433.06MB/s
  Status codes: 2xx=1071456, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 1071453 / 1071456 responses (100.0%)
  Reconnects: 42583
  Per-template: 213749,214443,214879,214608,213774
  Per-template-ok: 213749,214443,214879,214608,213774
[info] CPU 6332.1% | Mem 3.1GiB

=== Best: 108825 req/s (CPU: 6244.8%, Mem: 3.2GiB) ===
[info] input BW: 7.26MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/sinatra.json
[warn] sinatra async-db/1024: container log was 115734225 bytes, truncated to 8000000
httparena-bench-sinatra
httparena-bench-sinatra

==============================================
=== sinatra / crud / 4096c (tool=gcannon) ===
==============================================
[info] resetting postgres for a clean per-profile baseline
[info] starting postgres sidecar
httparena-postgres
[info] postgres ready (seeded)
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   65.97ms   66.30ms   75.70ms   91.50ms   309.00ms

  892219 requests in 15.00s, 888253 responses
  Throughput: 59.21K req/s
  Bandwidth:  18.67MB/s
  Status codes: 2xx=888253, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 888253 / 888253 responses (100.0%)
  Reconnects: 3215
  Per-template: 44701,44374,44022,44498,44498,44142,43949,44183,44316,44359,44438,44212,44014,44425,44663,44594,44247,46151,43986,44481
  Per-template-ok: 44701,44374,44022,44498,44498,44142,43949,44183,44316,44359,44438,44212,44014,44425,44663,44594,44247,46151,43986,44481
[info] CPU 3220.6% | Mem 3.7GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   64.41ms   66.50ms   73.00ms   83.20ms   277.20ms

  917346 requests in 15.00s, 914969 responses
  Throughput: 60.99K req/s
  Bandwidth:  18.97MB/s
  Status codes: 2xx=914969, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 914968 / 914969 responses (100.0%)
  Reconnects: 3609
  Per-template: 45613,46129,45654,45558,45511,45673,45863,45579,45503,45797,45735,45807,45358,45938,45653,45603,44956,47661,45681,45696
  Per-template-ok: 45613,46129,45654,45558,45511,45673,45863,45579,45503,45797,45735,45807,45358,45938,45653,45603,44956,47661,45681,45696
[info] CPU 2894.9% | Mem 3.9GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  200
  Templates: 20
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   65.14ms   67.10ms   73.60ms   84.30ms   259.10ms

  907328 requests in 15.00s, 905098 responses
  Throughput: 60.33K req/s
  Bandwidth:  18.72MB/s
  Status codes: 2xx=905098, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 905097 / 905098 responses (100.0%)
  Reconnects: 3469
  Per-template: 45479,45646,45291,44927,44837,45314,45106,45139,45293,45420,45209,45340,44860,45294,45150,45046,44161,47228,45400,44957
  Per-template-ok: 45479,45646,45291,44927,44837,45314,45106,45139,45293,45420,45209,45340,44860,45294,45150,45046,44161,47228,45400,44957
[info] CPU 2911.3% | Mem 3.9GiB

=== Best: 60997 req/s (CPU: 2894.9%, Mem: 3.9GiB) ===
[info] input BW: 5.24MB/s (avg template: 90 bytes)
[info] saved results/crud/4096/sinatra.json
httparena-bench-sinatra
httparena-bench-sinatra
[info] skip: sinatra does not subscribe to fortunes
[info] skip: sinatra does not subscribe to baseline-h2
[info] skip: sinatra does not subscribe to static-h2
[info] skip: sinatra does not subscribe to baseline-h2c
[info] skip: sinatra does not subscribe to json-h2c
[info] skip: sinatra does not subscribe to baseline-h3
[info] skip: sinatra does not subscribe to static-h3
[info] skip: sinatra does not subscribe to gateway-64
[info] skip: sinatra does not subscribe to gateway-h3
[info] skip: sinatra does not subscribe to production-stack
[info] skip: sinatra does not subscribe to unary-grpc
[info] skip: sinatra does not subscribe to unary-grpc-tls
[info] skip: sinatra does not subscribe to echo-ws
[info] skip: sinatra does not subscribe to echo-ws-pipeline
[info] skip: sinatra does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/sinatra.json - 23 new, 23 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

p8 requested review from Kaliumhexacyanoferrat and MDA2AV as code owners August 24, 2026 19:17

MDA2AV force-pushed the sinatra/simplify-baseline branch from b30385b to b09daf8 Compare August 24, 2026 23:09

[sinatra] Simplify total calculation for baseline11

978e577

This makes it consistent with other calls where we also sum the a and b params.

MDA2AV force-pushed the sinatra/simplify-baseline branch from b09daf8 to 978e577 Compare August 25, 2026 00:52

Benchmark results: sinatra [skip ci]

b528520

MDA2AV approved these changes Aug 25, 2026

View reviewed changes

MDA2AV merged commit b0a9c5c into MDA2AV:main Aug 25, 2026

Navigation Menu

Uh oh!

[sinatra] Simplify total calculation for baseline11 - #1306

[sinatra] Simplify total calculation for baseline11#1306
MDA2AV merged 2 commits into
MDA2AV:mainfrom
p8:sinatra/simplify-baseline

p8 commented Aug 24, 2026

Uh oh!

p8 commented Aug 24, 2026

Uh oh!

github-actions Bot commented Aug 24, 2026

Uh oh!

github-actions Bot commented Aug 24, 2026

Uh oh!

MDA2AV commented Aug 25, 2026

Uh oh!

github-actions Bot commented Aug 25, 2026

Uh oh!

github-actions Bot commented Aug 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

p8 commented Aug 24, 2026

Uh oh!

p8 commented Aug 24, 2026

Uh oh!

github-actions Bot commented Aug 24, 2026

Uh oh!

github-actions Bot commented Aug 24, 2026

Benchmark Results

Uh oh!

MDA2AV commented Aug 25, 2026

Uh oh!

github-actions Bot commented Aug 25, 2026

Uh oh!

github-actions Bot commented Aug 25, 2026

Benchmark Results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants