Repository navigation
perf(ws,realtime): offload blocking backplane publish + chat/upload I/O off the event loop - #234
Merged
Merged
Conversation
…/O off the loop Extends the event-loop-blocking fix (session pipeline, separate PR) to the WebSocket and realtime subsystems. The Redis backplane LISTENER already runs in a thread; these were still on the loop: - WS backplane PUBLISH: broadcast/broadcast_all/broadcast_to_room (async, on the loop) called _publish -> synchronous redis.publish inline. Every broadcast - a WS send, a WebRTC signalling relay, a chat fan-out - was a blocking socket round-trip ON the loop. Now awaited through asyncio.to_thread. - Realtime chat: _persist_message (blocking ORM write) and _mark_read's read-receipt read+write ran on the loop inside async handlers; offloaded. - Realtime files_upload: the blocking _store_upload (local/S3 storage write) ran on the loop; offloaded. WebRTC signalling itself is a pure relay (broadcast_to_room), so it is covered by the backplane offload with no other change. Proven (tests/test_ws_backplane_threading.py, no mocks - real WebSocketManager, real clock/threads, a real backplane with a measured publish latency): RED before, GREEN after. 4 concurrent broadcasts 1.22s (serialised) -> under 0.60*serial. Regression: 160 realtime + websocket tests pass. Python-only (PHP forks, Ruby threads per connection). NOT changed: a user's own sync WS message handler still runs on the loop - the async-handler convention and per-connection message order make threading it a deliberate non-goal, same honest limit ADR-0074 documents. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Andre van Zuydam <andrevanzuydam@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the event-loop audit (session pipeline is #233) to WebSocket, WebRTC and realtime. The Redis backplane's listener already runs in a thread; these three paths were still blocking the loop:
broadcast*()(async) →_publish→ synchronousredis.publishinline on the loop — every WS send / WebRTC signalling relay / chat fan-out a blocking socket round-tripawait asyncio.to_thread(self._publish, …)_persist_message(blocking ORM write) +_mark_read(read+write) on the loop in async handlersto_thread_store_upload(local/S3 storage write) on the loopto_threadWebRTC signalling is a pure relay (
broadcast_to_room), so the backplane offload covers it with no other change.Proven + measured (no mocks)
tests/test_ws_backplane_threading.pydrives a realWebSocketManagerwith a real clock/threads and a real backplane carrying a measured publish latency — RED before, GREEN after. 4 concurrent broadcasts: 1.22s (serialised) → under 0.6×serial. Regression: 160 realtime + websocket tests pass; metrics gate: no regression.Deliberately NOT changed
A user's own sync WS message handler still runs on the loop. The async-handler convention and per-connection message ordering make threading each message a non-goal — the same honest limit ADR-0074 documents for a blocking async HTTP handler. Python-only (PHP forks per request, Ruby is thread-per-connection).
Companion to #233 (session pipeline). SSE's async-generator path is already correct (
async for); its rarely-used sync-iterable path pullsnext()on the loop — noted, not changed here.🤖 Generated with Claude Code