Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Changelog

## [Unreleased]

## [0.13.0] - 2026-10-20
### Fixed
- **Predicts against wide or long contexts ran as one-row calls** (#47). Predict chunks were sized with the fit memory model, which grows with the training table; past ~3,500 x 500 no chunk "fit", chunking fell to one row, and a 100-row predict became 100 sequential model calls: 14.4 s (TabICLv2) and 6.3 s (TabPFN-3.5) instead of ~0.1 s on an A100, ~24 s and ~11 s on prod's A40s. Chunks are now sized by `PredictCostModel` (`memory/predict.py`), built from predict peaks measured under the serving configuration (`benchmarks/calibrate_predict.py`, A100, compiled TabICL, int8 kv; generated `memory/calibration_predict_a100.py`): a cost per test row and a fixed cost per call, both by width, plus a cost per training row. It bounds every measurement it was built from and assumes at most linear growth beyond them (affine in width past the widest, where it scales in proportion to width), and it applies only to the calibrated configuration (kv cache, int8, compiled TabICL; TabPFN). When a call's fixed cost alone exceeds the headroom, only the rows above it are sized against the budget instead of splitting to single rows, which only multiplied that fixed cost; with no headroom a predict still runs as the smallest chunks, and a real OOM still halves a chunk. `LIGHT_PREDICT_ROWS` is now `chunking.MAX_CHUNK_ROWS` (2,048), so a light predict is one call.
- **A killed worker could look alive to `close()`**, which then kept its host-memory reservations (and failed `test_workers_close_together_within_one_grace_period` intermittently in CI). The reader thread and `close()` both reaped a dying worker; when two threads call waitpid at once, the loser finds no child and CPython reports the process still running until the winner records its exit code. Every wait, liveness check and kill of a worker is now serialized under one lock (waits on its process sentinel happen outside it, so health checks never block), so every caller sees the exit, and its code, once one does; a sentinel a forked grandchild still holds falls back to a non-blocking check.
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -470,11 +470,12 @@ sharing one A100-40GB** (v0.7.0, 2026-08-29,

## Status & Roadmap

v0.12.0: multi-model endpoint, calibrated peak-aware admission, disk
v0.13.0: multi-model endpoint, calibrated peak-aware admission, disk
spillover, multi-replica routing and multi-GPU replicas with a worker
process per GPU have all shipped and been validated on real GPUs (see
[CHANGELOG.md](CHANGELOG.md)); 0.12.0 serves on Vector's 4x A40 prod
node. Open roadmap items are
[CHANGELOG.md](CHANGELOG.md)); 0.13.0 fixes predict chunking and
compiled TabICL in serving (#47). Vector's 4x A40 prod node serves
0.12.0 until it is updated. Open roadmap items are
tracked as GitHub issues labelled `roadmap`. **Read
[ROADMAP.md](ROADMAP.md) before starting new work** -- it ranks what's
next and why.
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "tabctx"
version = "0.12.0"
version = "0.13.0"
description = "Serving engine for tabular foundation models (TabICL, TabPFN): fit a table once, keep its encoded context on the GPU, and serve fast repeat predictions to many tenants, with memory-aware admission and multi-GPU placement."
readme = "README.md"
license = "Apache-2.0"
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading