Repository navigation
docs: redraw architecture diagram for 0.12.0 (multi-GPU router, worker processes, host guard) - #53
Merged
Merged
Conversation
…ses, host guard The figure predated 0.12.0: it showed one TabctxService replica per GPU share with a single engine lock, and had no MultiDeviceEngine, per-GPU worker processes, residency events, host-memory guard or offload tier. Redraw it from the code, update the caption, and fix the README's coalescing claim (light predicts now run beside fits). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guard scope, contrast) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #53 +/- ##
==========================================
+ Coverage 78.72% 78.92% +0.19%
==========================================
Files 39 39
Lines 3098 3098
==========================================
+ Hits 2439 2445 +6
+ Misses 659 653 -6
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The architecture figure in the README predated 0.12.0 (#38, #40). This PR redraws
docs/figures/architecture.svgfrom the code (still hand-editable SVG, same palette, explicit white background so it reads the same in GitHub light and dark themes), updates the caption, and fixes one stale README claim. Rendered with cairosvg and checked for overlaps and clipping.Discrepancies found (old → new, with evidence)
Diagram
serve/factory.py:692-744,resolve_devices;TABCTX_DEVICES=auto).PlacementPolicy/LeastActiveThatFits, per-dataset_id locks, re-fit ordering, lock-free snapshots (multidevice.py:1-40, 148-175;placement.py:53-80).ProcessShard, a pipe per worker, and a Snapshot on every reply. Dead workers fail the health check (serve/workers.py:1-37,factory.py:746-797,app.py:514-520).engine.py:28-34, 186-198).factory.py:465-505,memory/requirement.py:1-30,engine.py:401-422)./v1/tabctx/residency→ shown on the router (events.py,app.py:538-568).memory/host.py:1-45,factory.py:757-770).factory.py:547-558, 597-616).factory.py:508-544, CHANGELOG 0.12.0)./v1/tabctx/residency(app.py:538). The factory box now listsTABCTX_DEVICES,_WORKER_PROCESSES,_KV_PRECISIONand_COMPILE, plus host-budget detection.README
batching.py:39-45).Checked and left unchanged because they are correct: "Scaling out", "Several GPUs in one replica", "Large tables", "Tenant isolation" (against
affinity.py,uploads.py,tenancy.py,placement.py,app.py).Not changed (out of scope): "Status & Roadmap" still says v0.10.0.
pre-commit run --all-filespasses.Do not merge.
🤖 Generated with Claude Code