Skip to content

Address JOSS review feedback (#147, #148, #149) - #150

Merged
TomeHirata merged 1 commit into
mainfrom
ccr-e30220fe-ga9flf
Oct 6, 2026
Merged

TomeHirata merged 1 commit into
mainfrom
ccr-e30220fe-ga9flf

Conversation

@TomeHirata

@TomeHirata TomeHirata commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Addresses the issues raised in the JOSS review (openjournals/joss-reviews#11047).

Changes

  • Outcomes shape handling in predict_dte #147 – outcome shape handling: all fit methods now flatten single-column treatment_arms, outcomes, strata and treatment_indicator (e.g. shape (n, 1)) to 1-D and raise a clear ValueError for other shapes. predict_dte now returns identical results for Y and Y[:, None].
  • Statistical validation with known baselines #148 – QTE bug: predict_qte returned wrong values (e.g. 1 instead of 5 for a constant shift of 5) because the quantile search returned the largest location with F(y) <= q. It now returns the generalized inverse, the smallest y with F(y) >= q. Added tests/test_statistical_baselines.py with hand-computed baselines (DTE/PTE/QTE), a stratified-vs-simple equivalence check, and a simulated normal location-shift DGP with known DTE/QTE for the simple and adjusted estimators (including a CI-width check for the adjusted one). The existing mock-based test_predict_qte relied on the buggy behaviour and now uses a CDF mock with a known quantile shift.
  • Confounding adjustment language in docstrings #149 – confounding language: docstrings of the adjusted estimators, the docstring examples (now randomized treatment) and the Oregon tutorial now describe ML adjustment as variance reduction for randomized experiments, not confounding correction.
  • Minor points
    • Empty folds: cross-fitting now raises an informative ValueError suggesting fewer folds when a fold leaves no training data for the target arm.
    • Missing values: fit rejects NaN in treatment_arms, outcomes, strata and treatment_indicator; covariates are not checked (depends on the base model). Documented in the get-started guide.

Not included

Bootstrap inference options for the partial-compliance (predict_ldte/predict_lpte) methods. This is a new statistical feature rather than a fix and needs a design that respects the stratified sampling scheme, so I left it for a follow-up.

Testing

pytest (72 tests) and ruff check . pass.

- Validate/flatten fit inputs: accept single-column outcomes, treatment arms,
  strata and treatment indicator; reject other shapes and NaN in them.
- Fix QTE: use the generalized inverse of the CDF (smallest y with F(y) >= q).
- Raise an informative error when cross-fitting leaves no training data and
  suggest reducing folds.
- Clarify that ML adjustment targets variance reduction in randomized
  experiments, not confounding correction (docstrings, examples, docs).
- Add shape and statistical-baseline tests; document missing-value policy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LjhUSYBaHyrPdnQ7z13FKv
Copilot AI balanced review requested due to automatic review settings October 6, 2026 09:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@TomeHirata
TomeHirata merged commit 1aeb951 into main Oct 6, 2026
10 checks passed
@TomeHirata
TomeHirata deleted the ccr-e30220fe-ga9flf branch October 6, 2026 11:31
@TomeHirata TomeHirata changed the title Address JOSS review feedback from @msukiasyan (#147, #148, #149) Address JOSS review feedback (#147, #148, #149) Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants