Skip to content

Collapse cohort layer into one SpatialDataCohort; version-pinned freeze/manifest - #20

Merged
timtreis merged 16 commits into
mainfrom
feat/cohort-redesign
Oct 15, 2026
Merged

timtreis merged 16 commits into
mainfrom
feat/cohort-redesign

Conversation

@timtreis

Copy link
Copy Markdown
Member

Summary

Collapses the client's two cohort classes into one immutable SpatialDataCohort, reduces "frozen" to a serializable CohortManifest, pins cohorts to exact full-uid dataset versions, and cuts the downstream task-adapter execution code out of the client. Breaking change to the cohort surface (client is 0.x).

What changed

One cohort object (cohort.py)

  • freeze() → offline CohortManifest (identity = sha256 over the members' full lamin uids + canonical filter; no content fingerprint, no network). from_manifest() rehydrates without a catalog re-query.
  • Immutable subsetting: filter(**facets), where(mask), | (union, deduped by uid), - (difference) — all return new cohorts.
  • verify() = liveness check (pinned store resolves + opens); check_updates(catalog) = stem-join supersession detection.
  • download(dest, *, strict=False) — resumable; a missing pinned store falls back to the latest version with a loud warning, or raises CohortDriftError under strict / when no fallback resolves.
  • split(...) → {fold: SpatialDataCohort} (group-safe, deterministic), each carrying the derived SplitManifest.
  • to_anndata() → outer join with obs['uid'] + obs['study_id'].
  • sdatas / load(elements=...) for explicit object loading; Dataset.load(elements=...).

Catalog / schema

  • Catalog.cohort(uids=[...] | study_ids=[...]) explicit construction.
  • study_id and doi are now query facets (doi is exempt from free-text search to avoid DOI-fragment false matches).

Manifest (manifest.py)

  • New CohortManifest identity record (JSON, with a parquet sidecar for large member sets) alongside the existing resumable download Manifest.

Removals

  • FrozenCohort / freeze.py, and build / materialize / check / TaskView / PreflightReport + element-resolution machinery from adapter.py — the client keeps only the TaskAdapter Protocol (adapter execution is downstream). Removal canary tests enforce un-importability.

Version model (context)

A frozen cohort is pinned to the exact full lamin uid@version. Byte-exact retrieval of superseded versions depends on an engine-side change (retain each version's .zarr via overwrite_versions=False) that is release-gated and out of scope here; the client is forward-compatible against that contract. See the design spec for the full version model.

Testing

Full suite green: 284 passed, 4 skipped, 5 network-deselected. ruff check clean. TDD throughout; a whole-branch review added non-mocked tests for the download missing-store path.

Follow-ups (non-blocking)

  • __or__/__sub__ could return NotImplemented for a non-cohort operand.
  • Download fallback warning stacklevel; slicing a split fold drops .split_manifest.

…FrozenCohort + build/materialize; update exports
…arch exempt, split mode, empty-cohort + set-filter round-trip
@codecov

codecov Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.89340% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.00%. Comparing base (f9a8a9b) to head (b0af64a).

Files with missing lines Patch % Lines
src/sddb/manifest.py 90.10% 9 Missing ⚠️
src/sddb/cohort.py 96.25% 3 Missing ⚠️
src/sddb/catalog.py 87.50% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #20      +/-   ##
==========================================
+ Coverage   91.91%   92.00%   +0.08%     
==========================================
  Files          19       18       -1     
  Lines        1460     1450      -10     
==========================================
- Hits         1342     1334       -8     
+ Misses        118      116       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@timtreis
timtreis merged commit 2335fb6 into main Oct 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant