Repository navigation
perf(server): offload blocking session load/save off the event loop - #233
Merged
Merged
Conversation
ADR-0074 threaded a blocking route HANDLER so it no longer froze the asyncio loop, but only the handler was offloaded - every dispatch STAGE still ran on the loop. Two of them do blocking I/O: session LOAD (_init_session -> handler.read) and session SAVE (_stage_session_save -> handler.write). With a network session backend (Redis/Valkey/memcached/MongoDB) each is a blocking socket round-trip ON the loop, so even a fully async route serialised every request through its session I/O; the file backend blocks on a local read/write. Offload both via asyncio.to_thread (the same tool ADR-0074 used for handlers): the LOAD in handle(), and the SAVE + the request-log file write through a new _run_response_stage helper so the dispatch runner stays flat (the god-function guard). The Session and its backend connection are request-local, so a worker thread adds no cross-request sharing. Proven (tests/test_dispatch_stage_threading.py, no mocks - real dispatch, real clock/threads, a real file-backed handler with a measured latency): both tests are RED before and GREEN after. Measured with 0.3s load + 0.3s save latency, 4 concurrent requests: 2.44s (serialised) -> 0.62s (~3.9x), scaling with concurrency. Python-only: PHP forks per request and Ruby is thread-per-connection, so neither has a shared loop to block. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Andre van Zuydam <andrevanzuydam@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
ADR-0074 threaded a blocking route handler so it no longer froze the asyncio loop — but only the handler. Every dispatch stage still runs on the loop (
handle()incore/server.py), and two of them do blocking I/O:_init_session→Session.start(sid)→handler.read_stage_session_save→session.save()→handler.writeWith a network session backend (Redis/Valkey/memcached/MongoDB) each is a blocking
socketround-trip on the loop, so even a fullyasyncroute serialises every request through its session I/O (thefilebackend blocks on a local read/write).The fix
Offload both with
asyncio.to_thread— the same tool ADR-0074 used for handlers:handle();_run_response_stagehelper, so the dispatch runner stays flat (the god-function guard intest_dispatch_pipeline.py).The
Sessionand its backend connection are request-local, so a worker thread adds no cross-request sharing.Proven + measured (no mocks)
tests/test_dispatch_stage_threading.pydrives the realserver.handledispatch with a real clock, real threads, and a real file-backed session handler carrying a measured latency (the ADR-0074 instrument, not a mock of a backend). Both tests are RED before, GREEN after:Scales with concurrency. Regression-checked: dispatch pipeline, ADR-0074 handler threading, security-headers every-response, request-id, concurrent-session, session/cookie/CSRF units — 197 green locally; metrics gate: no regression.
Python-only: PHP forks per request and Ruby is thread-per-connection, so neither has a shared loop to block — no parity change needed.
🤖 Generated with Claude Code