Skip to content

perf(server): offload blocking session load/save off the event loop - #233

Merged
tina4stack merged 1 commit into
v3from
fix/dispatch-stage-event-loop-blocking
Oct 15, 2026
Merged

tina4stack merged 1 commit into
v3from
fix/dispatch-stage-event-loop-blocking

Conversation

@tina4stack

Copy link
Copy Markdown
Owner

The gap

ADR-0074 threaded a blocking route handler so it no longer froze the asyncio loop — but only the handler. Every dispatch stage still runs on the loop (handle() in core/server.py), and two of them do blocking I/O:

  • session load — _init_session → Session.start(sid) → handler.read
  • session save — _stage_session_save → session.save() → handler.write

With a network session backend (Redis/Valkey/memcached/MongoDB) each is a blocking socket round-trip on the loop, so even a fully async route serialises every request through its session I/O (the file backend blocks on a local read/write).

The fix

Offload both with asyncio.to_thread — the same tool ADR-0074 used for handlers:

  • the load in handle();
  • the save + the request-log file write via a new _run_response_stage helper, so the dispatch runner stays flat (the god-function guard in test_dispatch_pipeline.py).

The Session and its backend connection are request-local, so a worker thread adds no cross-request sharing.

Proven + measured (no mocks)

tests/test_dispatch_stage_threading.py drives the real server.handle dispatch with a real clock, real threads, and a real file-backed session handler carrying a measured latency (the ADR-0074 instrument, not a mock of a backend). Both tests are RED before, GREEN after:

per-request session I/O 4 concurrent requests
before 0.6s (0.3 load + 0.3 save) 2.44s (serialised)
after 0.6s 0.62s (~3.9×)

Scales with concurrency. Regression-checked: dispatch pipeline, ADR-0074 handler threading, security-headers every-response, request-id, concurrent-session, session/cookie/CSRF units — 197 green locally; metrics gate: no regression.

Python-only: PHP forks per request and Ruby is thread-per-connection, so neither has a shared loop to block — no parity change needed.

🤖 Generated with Claude Code

ADR-0074 threaded a blocking route HANDLER so it no longer froze the asyncio
loop, but only the handler was offloaded - every dispatch STAGE still ran on the
loop. Two of them do blocking I/O: session LOAD (_init_session -> handler.read)
and session SAVE (_stage_session_save -> handler.write). With a network session
backend (Redis/Valkey/memcached/MongoDB) each is a blocking socket round-trip ON
the loop, so even a fully async route serialised every request through its
session I/O; the file backend blocks on a local read/write.

Offload both via asyncio.to_thread (the same tool ADR-0074 used for handlers):
the LOAD in handle(), and the SAVE + the request-log file write through a new
_run_response_stage helper so the dispatch runner stays flat (the god-function
guard). The Session and its backend connection are request-local, so a worker
thread adds no cross-request sharing.

Proven (tests/test_dispatch_stage_threading.py, no mocks - real dispatch, real
clock/threads, a real file-backed handler with a measured latency): both tests
are RED before and GREEN after. Measured with 0.3s load + 0.3s save latency, 4
concurrent requests: 2.44s (serialised) -> 0.62s (~3.9x), scaling with
concurrency. Python-only: PHP forks per request and Ruby is thread-per-connection,
so neither has a shared loop to block.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Andre van Zuydam <andrevanzuydam@gmail.com>
@tina4stack
tina4stack merged commit 996efd5 into v3 Oct 15, 2026
14 checks passed
@tina4stack
tina4stack deleted the fix/dispatch-stage-event-loop-blocking branch October 15, 2026 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants