Skip to content

fix(predict): classification labels are strings whatever return_proba is (#45) - #56

Merged
amrit110 merged 2 commits into
mainfrom
fix/label-types
Oct 20, 2026
Merged

amrit110 merged 2 commits into
mainfrom
fix/label-types

Conversation

@amrit110

Copy link
Copy Markdown
Member

Closes #45.

Problem

A classification's predictions changed type with return_proba:

return_proba: true return_proba: false
tabicl-v2 (before) ["1.0"] [1.0]
tabpfn-3.5 ["1.0"] ["1.0"]

Without probabilities, TabICLBackend._predict returned model.predict(X).tolist(). That keeps the label dtype, and it goes against the backend protocol, which says to derive labels from probabilities.

Fix

  • New backends.base.classification_outcome(proba, classes, return_proba): argmax over [str(c) for c in classes], with probabilities and classes only when asked. TabICL and TabPFN both use it, so the two backends can't drift apart again, and TabICL no longer calls predict().
  • API schema: the predict responses' predictions field documents the contract. Classification gives strings whatever return_proba is; regression gives numbers.
  • Behaviour change: tabicl-v2 clients that relied on numeric labels without return_proba now get strings, as with tabpfn-3.5. This is noted in the CHANGELOG.

Tests

  • New tests/unit/test_classification_outcome.py:
    • The helper's contract.
    • Both real backends' _predict, driven by a stub estimator whose predict() raises. The TabICL case fails on main's code (checked).
  • Unit tests: 501 pass.

Not in this PR

No architecture change.

🤖 Generated with Claude Code

@amrit110 amrit110 added bug Something isn't working P1 High: next up area/api Request and response contract, errors, uploads labels Oct 20, 2026
@amrit110

Copy link
Copy Markdown
Member Author

Review fixes pushed. The independent review found no bugs: tabicl 2.1.1's and tabpfn 8.5.0's predict() are exactly argmax(predict_proba) over classes_, so the labels are unchanged, and only the type changes.

  • Helper: moved to backends/_outcome.py, so base.py stays protocol-only.
  • Fake backend: reports through the same helper. On ties its label is now the argmax of its own probabilities.
  • Real-model check: tests/integration/test_tabicl_kv_real.py now also asserts that return_proba=False gives the same labels, drawn from classes. This runs on the GPU only.
  • Wording: API doc and CHANGELOG.

Unit tests: 501 pass.

amrit110 and others added 2 commits October 20, 2026 17:20
… is (#45)

TabICL returned predict()'s labels without probabilities (numbers for a
JSON train_y) and strings from classes with them; TabPFN always returned
strings. Both now go through backends.base.classification_outcome: one
predict_proba pass, argmax over the class strings, probabilities and
classes only when asked. The predictions field documents the contract.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…igned

- classification_outcome moves to backends/_outcome.py, keeping base.py
  protocol-only; proba typed ArrayLike.
- The fake backend reports through it too, so its label is the argmax of
  its own probabilities (ties went to the first label seen).
- The real-model kv test checks return_proba=False gives the same labels.
- API doc and CHANGELOG wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 79.45%. Comparing base (7579da1) to head (26e28a3).

Files with missing lines Patch % Lines
src/tabctx/backends/fake.py 85.71% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #56      +/-   ##
==========================================
+ Coverage   79.43%   79.45%   +0.02%     
==========================================
  Files          41       42       +1     
  Lines        3209     3218       +9     
==========================================
+ Hits         2549     2557       +8     
- Misses        660      661       +1     
Flag Coverage Δ
integration 30.08% <18.18%> (-0.06%) ⬇️
unit 75.94% <77.27%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/tabctx/backends/_outcome.py 100.00% <100.00%> (ø)
src/tabctx/backends/base.py 100.00% <ø> (ø)
src/tabctx/serve/app.py 35.87% <100.00%> (+0.20%) ⬆️
src/tabctx/backends/fake.py 98.14% <85.71%> (-1.86%) ⬇️

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@amrit110
amrit110 merged commit 9afb5ab into main Oct 20, 2026
3 checks passed
@amrit110 amrit110 mentioned this pull request Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/api Request and response contract, errors, uploads bug Something isn't working P1 High: next up

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TabICL classification predict returns numeric labels without return_proba, strings with it

2 participants