Skip to content

docs: redraw architecture diagram for 0.12.0 (multi-GPU router, worker processes, host guard) - #53

Merged
amrit110 merged 3 commits into
mainfrom
docs/architecture-diagram
Oct 20, 2026
Merged

amrit110 merged 3 commits into
mainfrom
docs/architecture-diagram

Conversation

@amrit110

Copy link
Copy Markdown
Member

The architecture figure in the README predated 0.12.0 (#38, #40). This PR redraws docs/figures/architecture.svg from the code (still hand-editable SVG, same palette, explicit white background so it reads the same in GitHub light and dark themes), updates the caption, and fixes one stale README claim. Rendered with cairosvg and checked for overlaps and clipping.

Discrepancies found (old → new, with evidence)

Diagram

  1. Replica box said "one per GPU share". A replica now serves one or more GPUs (serve/factory.py:692-744, resolve_devices; TABCTX_DEVICES=auto).
  2. No multi-GPU router → added a MultiDeviceEngine layer: PlacementPolicy/LeastActiveThatFits, per-dataset_id locks, re-fit ordering, lock-free snapshots (multidevice.py:1-40, 148-175; placement.py:53-80).
  3. No per-GPU worker processes → added a stacked worker process per GPU with ProcessShard, a pipe per worker, and a Snapshot on every reply. Dead workers fail the health check (serve/workers.py:1-37, factory.py:746-797, app.py:514-520).
  4. "one lock serializes all GPU work per replica" → each GPU's engine runs one fit at a time, and predicts of up to 2,048 rows on a resident context run beside it (engine.py:28-34, 186-198).
  5. Admission "estimated fit PEAK must fit 85%..." → estimate is resident + working floor from calibrated requirement grids where available, plus a check of real free device memory (factory.py:465-505, memory/requirement.py:1-30, engine.py:401-422).
  6. No residency events or /v1/tabctx/residency → shown on the router (events.py, app.py:538-568).
  7. Host-memory guard missing → added a HostMemoryGuard shared by every worker on the node (memory/host.py:1-45, factory.py:757-770).
  8. Tiers were missing the TabICL offload dir and the per-GPU spill subdirectories (factory.py:547-558, 597-616).
  9. Backends box was missing int8 KV, compiled attention, GPU Yeo-Johnson, and TabPFN's shared model (factory.py:508-544, CHANGELOG 0.12.0).
  10. Endpoint list was missing /v1/tabctx/residency (app.py:538). The factory box now lists TABCTX_DEVICES, _WORKER_PROCESSES, _KV_PRECISION and _COMPILE, plus host-budget detection.

README

  • Caption now describes the router, worker processes and host guard, and notes that a single-GPU replica calls its one engine directly.
  • "How it works", coalescing: "without ever running concurrent GPU work (single-in-flight-call assumption)" → "without adding concurrent GPU calls (each GPU still runs at most one predict at a time, beside at most one fit)" (batching.py:39-45).
  • "How it works" had no entries for multi-GPU or host-memory admission. Added two short bullets.

Checked and left unchanged because they are correct: "Scaling out", "Several GPUs in one replica", "Large tables", "Tenant isolation" (against affinity.py, uploads.py, tenancy.py, placement.py, app.py).

Not changed (out of scope): "Status & Roadmap" still says v0.10.0.

pre-commit run --all-files passes.

Do not merge.

🤖 Generated with Claude Code

amrit110 and others added 3 commits October 20, 2026 13:50
…ses, host guard

The figure predated 0.12.0: it showed one TabctxService replica per GPU
share with a single engine lock, and had no MultiDeviceEngine, per-GPU
worker processes, residency events, host-memory guard or offload tier.
Redraw it from the code, update the caption, and fix the README's
coalescing claim (light predicts now run beside fits).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guard scope, contrast)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.92%. Comparing base (bfac40a) to head (1318489).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
+ Coverage   78.72%   78.92%   +0.19%     
==========================================
  Files          39       39              
  Lines        3098     3098              
==========================================
+ Hits         2439     2445       +6     
+ Misses        659      653       -6     
Flag Coverage Δ
integration 30.05% <ø> (ø)
unit 75.30% <ø> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@amrit110
amrit110 merged commit 6e386d8 into main Oct 20, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants