It would be good to add more tests covering statistical correctness of the methods directly. This could be against simple baselines with hand-computed outcomes or slightly more complex simulations with known treatment effects in the DGP. I see such simulations in the underlying papers but having tests like that here could catch implementation bugs. I did find one such bug in a simple case for QTE where treatment effects are 5 everywhere but requesting the median returns 1, see below.
from dte_adj import SimpleDistributionEstimator
np.random.seed(0)
D = np.repeat([0, 1], 20)
X = np.zeros((40, 1))
Y = 5.0 * D
est = SimpleDistributionEstimator().fit(X, D, Y)
qte, _, _ = est.predict_qte(
1, 0, quantiles=np.array([0.5]), n_bootstrap=5, display_progress=False
)
np.testing.assert_allclose(qte, [5.0], rtol=0, atol=1e-10)
# AssertionError: actual [1.], desired [5.]```
This issue is in relation to JOSS review thread openjournals/joss-reviews#11047
It would be good to add more tests covering statistical correctness of the methods directly. This could be against simple baselines with hand-computed outcomes or slightly more complex simulations with known treatment effects in the DGP. I see such simulations in the underlying papers but having tests like that here could catch implementation bugs. I did find one such bug in a simple case for QTE where treatment effects are 5 everywhere but requesting the median returns 1, see below.