Improve HTTP/2 and HTTP/3 multiplexing, add stable benchmark suite, and carry forward upstream-aligned fixes

[menudoproblema]

This PR bundles a runtime update together with a maintainable benchmark suite that can live in main.

At a high level, it does three things:

1. Removes application-level HoL blocking for HTTP/2 by decoupling per-stream delivery from the connection loop, and applies the same scheduling structure to HTTP/3.
2. Improves several hot paths with low-risk runtime changes:
   • cached common response headers,
   • reduced redundant byte copies,
   • smaller per-event / per-buffer objects,
   • tighter H3 send batching,
   • cheaper access-log atom construction.
3. Fixes several correctness and robustness issues found while validating the runtime changes:
   • uninitialized HTTP and WebSocket stream state on out-of-order events,
   • Trio event clearing that could strand waiters,
   • H3 scheduler shutdown behavior,
   • oversized per-stream queue admission in QueuedStream,
   • unnecessary asyncio.wait_for() wrapping when no read timeout is configured,
   • fixed-size asyncio QUIC receive queue now made configurable.
4. Turns benchmarks/ into a reusable suite instead of a loose collection of scripts:
   • interleaved current/baseline runs by default,
   • median-of-runs summaries by default,
   • shared support for worktrees, server lifecycle, TLS and JSON output,
   • dedicated scenarios for general HTTP, H2 HoL, fragmented H2 bodies, HTTP/3 and WebSocket echo.

Important note about previously incorporated PRs

This branch already carries a small set of upstream-aligned PRs that were incorporated earlier into the fork and are not being presented here as the net-new value of this PR:

• Fix WSGI Violations #332: Fix WSGI violations
• Bugfix handle already-closed on DataReceived #334: handle already-closed on DataReceived
• Handle TimeoutError when waiting for the writer to close #342: handle TimeoutError while waiting for the writer to close
• chore: remove unused mock dev dependency #343: remove unused mock dev dependency

The performance numbers below therefore compare the current HEAD against upstream/main, but they should be read as the result of the current fork state as a whole, not as a claim that every line of improvement is newly introduced by this PR alone.

What changed

Runtime

• HTTP/2:
  • decoupled stream delivery from the connection loop,
  • added an explicit H2 send scheduler,
  • improved per-stream buffering and backpressure behavior,
  • reduced per-buffer object overhead.
• HTTP/3:
  • added a send scheduler and batching,
  • fixed scheduler shutdown behavior,
  • tuned batching and closure behavior for a better latency/robustness balance.
• Shared/runtime improvements:
  • cached common response headers,
  • removed redundant copies in several request/response paths,
  • reduced object overhead with slots in hot-path event/buffer types,
  • optimized access-log atom generation,
  • removed unnecessary asyncio.wait_for() overhead when no read timeout is configured,
  • made the asyncio QUIC receive queue configurable instead of using a fixed small buffer.

Correctness fixes

• initialized HTTPStream state defensively so Body, EndBody, and StreamClosed cannot hit missing attributes before Request;
• initialized WSStream handshake/connection state defensively for out-of-order event safety;
• fixed Trio EventWrapper.clear() so existing waiters are not stranded on a replaced event object;
• made H3 scheduler closure explicit so new operations fail fast after shutdown instead of waiting indefinitely;
• fixed QueuedStream admission so a first oversized event does not deadlock against max_queue_bytes.

Benchmarks

• extracted shared benchmark infrastructure into internal support modules;
• standardized comparison methodology across scenarios;
• documented the suite as project tooling suitable for main;
• fixed the benchmark-side TLS/ALPN mismatch for WebSocket-over-TLS measurements.

Benchmark methodology

All public numbers below were produced against upstream/main using the benchmark suite in this branch with the following rules:

• interleaved current and baseline runs;
• median-of-runs summary, not a single linear aggregation;
• dedicated scenarios reported separately instead of collapsing everything into one number.

This is important because earlier sequential runs were noisy enough to produce misleading conclusions in HTTP/1.1. The current methodology is intended to minimize order effects and machine drift.

Improvement percentages below are relative to upstream/main:

• positive values mean better throughput or lower latency;
• negative values mean worse throughput or higher latency.

Commands used:

python -m benchmarks.compare_general \
  --baseline-ref upstream/main \
  --runs 6 \
  --concurrency 40 \
  --total-requests 700 \
  --warmup-requests 80

python -m benchmarks.compare \
  --baseline-ref upstream/main \
  --runs 6 \
  --warmup-iterations 2 \
  --measured-iterations 12 \
  --fast-streams 5 \
  --slow-chunks 12 \
  --slow-delay-ms 25

python -m benchmarks.compare_fragmented_body \
  --baseline-ref upstream/main \
  --runs 6 \
  --warmup-iterations 2 \
  --measured-iterations 12 \
  --chunks 256 \
  --chunk-size 128 \
  --delay-ms 1

python -m benchmarks.compare_h3 \
  --baseline-ref upstream/main \
  --runs 6 \
  --concurrency 40 \
  --total-requests 400 \
  --warmup-requests 60

Results vs upstream/main

General HTTP

Scenario	Throughput	Mean latency	p95 latency	Reading
HTTP/1.1	-0.95% req/s	-1.03%	-2.79%	near-neutral, slightly worse
HTTP/1.1 + TLS	+0.41% req/s	+0.37%	-1.14%	near-neutral, mixed result
HTTP/2	+9.62% req/s	+8.58%	+0.77%	clear throughput/mean improvement, near-neutral tail latency

Raw medians-of-runs:

• HTTP/1.1:
  • current: 314.48 req/s, 124.57 ms mean, 340.46 ms p95
  • upstream: 317.50 req/s, 123.31 ms mean, 331.23 ms p95
• HTTP/1.1 + TLS:
  • current: 282.96 req/s, 138.79 ms mean, 378.16 ms p95
  • upstream: 281.81 req/s, 139.30 ms mean, 373.88 ms p95
• HTTP/2:
  • current: 2816.19 req/s, 14.02 ms mean, 16.71 ms p95
  • upstream: 2569.14 req/s, 15.34 ms mean, 16.84 ms p95

Targeted protocol scenarios

These scenarios are the most important for the structural runtime changes in this PR.

Scenario	Throughput	Mean latency	p95 latency	Reading
HTTP/2 HoL	n/a	+95.86%	+95.51%	very large improvement
HTTP/2 fragmented request body	n/a	+98.86%	+98.86%	very large improvement
HTTP/3 over QUIC	-23.54% req/s	-29.60%	+32.60%	mixed result: lower tail latency, worse throughput/mean

Raw medians-of-runs:

• HTTP/2 HoL:
  • current: 1.13 ms mean, 1.25 ms p95
  • upstream: 27.18 ms mean, 27.79 ms p95
• HTTP/2 fragmented request body:
  • current: 3.45 ms mean, 3.55 ms p95
  • upstream: 303.18 ms mean, 311.52 ms p95
• HTTP/3:
  • current: 1998.00 req/s, 18.91 ms mean, 21.25 ms p95
  • upstream: 2613.21 req/s, 14.59 ms mean, 31.53 ms p95

Interpretation

The main outcome is not a uniform micro-optimization across every path. The strongest result in this branch is still the same: Hypercorn becomes much better at the HTTP/2 cases where multiplexing actually matters:

• The benchmarked application-level HoL scenario in HTTP/2 is dramatically reduced.
• HTTP/2 remains clearly better in general throughput and mean latency, with near-neutral p95.
• HTTP/3 is mixed in the current public runs: p95 improves, but throughput and mean latency do not.

HTTP/1.1 remains close to upstream:

• plain HTTP/1.1 is slightly worse, but still near-neutral overall;
• HTTP/1.1 + TLS is also near-neutral overall, with slightly better throughput/mean and slightly worse p95.

That tradeoff is acceptable for this branch because the dominant gains are large, targeted and repeatable in HTTP/2, while the general HTTP/1.1 results remain close to upstream rather than materially diverging.

Why include the benchmark suite in the same PR

This branch has been developed under a strict “measure first” rule. Keeping the benchmark suite alongside the runtime changes makes future work easier to validate:

• contributors can compare against upstream/main or another local repo directly;
• the default methodology avoids the sequential-run noise that previously distorted conclusions;
• new performance changes can be judged against the same scenarios and guard-rails.

Suggested reviewer focus

• correctness of the H2/H3 per-stream delivery model and send schedulers;
• backpressure and closure behavior in multiplexed protocols;
• whether the benchmark suite is acceptable as maintained project tooling in main;
• whether the mixed-but-near-neutral HTTP/1.1 results are an acceptable tradeoff given the size of the HTTP/2 wins.

[XChikuX]

Thank you for this!

[rm-you]

Thanks for the thorough work here — this LOOKS very good, though I still am in the process of testing it. I had run into problems with multiplexing during some OpenStack work that this should resolve, so wanted to post just to express my interest. Really only had the one nit otherwise!

[rm-you]

Design nit: calling connection.send_headers() directly from the ASGI task bypasses the scheduler. All other h2.Connection mutations for sending (send_data, end_stream) are serialized through the single scheduler task in run() — this is the exception.

Suggested fix (~10 lines): store trailer headers on the StreamBuffer alongside the completion flag, then emit send_headers(..., end_stream=True) from _send_data when the buffer is complete and has trailers — same path that currently handles bare end_stream.

[menudoproblema]

Good catch. The trailers path should be serialized through the scheduler. drain() only indicates that the payload buffer has become empty; particularly with Trio, the waiting ASGI task may resume before the scheduler has completed the final send_data() mutation. Calling send_headers(..., end_stream=True) directly could therefore overtake the final DATA frame.

I’ll store the trailing headers on StreamBuffer, mark the buffer complete, and let _send_data() emit either trailing HEADERS or a bare end_stream after the payload has been drained. I’ll also add a regression test asserting DATA-before-trailers ordering.

[rm-you]

The all(stream.idle ...) check was removed — guessing because with QueuedStream + the send scheduler, stream-level idle doesn't capture whether the scheduler still has buffered frames. Is that actually the reasoning behind this change or is it something the AI agent I'm co-reviewing with made up? 😅

[menudoproblema]

Yes, broadly, although buffered scheduler frames are only part of the reasoning. After introducing QueuedStream and the separate send scheduler, stream.idle only describes application-delivery state; it no longer represents the complete HTTP/2 stream lifecycle. In particular, a WebSocket stream may report itself idle while its HTTP/2 stream is still registered or half-closed. _close_stream() is the point where both the registered stream and its scheduler state are removed, so the connection is now considered idle only when no registered streams remain. This avoids emitting Updated(idle=True) and starting the connection idle timeout prematurely.

This behavior is covered by test_protocol_idle_requires_no_registered_streams, although the rationale should be made clearer in the code.