[avehtari]

Summary

On CRAN, projpred's test setup fits no reference models. Thus no test of the submodel fitters from lme4, MASS, gamm4, ordinal and mclogit runs there (test_div_minimizer.R runs 0 of 3 tests), and the reverse dependency checks of these packages cannot see a change that affects projpred. Example: lme4 2.1-0 changed the fits of binomial submodels (lme4/lme4#1011). This PR adds tests that run on CRAN in about 2 seconds without Stan fits. It also filters the warning that lme4 will give for every Poisson submodel with group-level terms once lme4/lme4#1012 is released. With options(projpred.PQL = TRUE), it removes three warnings that every projection gives. See #207.

This PR was assisted by Claude.

Changes

• tests/testthat/test_submodel_fitters.R (new). The tests call divmin() and divmin_augdat() with small simulated data (6 groups of 8 observations). They test properties that the projection needs, not equality with a direct call of the same fitter.

  • lme4::glmer():
    • A binomial submodel with non-integer responses and a singular fit gives the glm() solution (nAGQ 1 and 20).
    • The traditional form and the augmented-data form (0/1 responses with weights mu and 1 - mu) of a binomial submodel give the same fit.
    • A Poisson submodel with non-integer responses estimates the group-level standard deviation (theta moves from its start value 1).
    • Observations with weight 0 do not change a binomial fit (nAGQ 1 and 10).
    • Integer weights give the same fit as duplicated observations (binomial and Poisson).
  • MASS::glmmPQL(): a Gaussian submodel gives the maximum-likelihood lmer() fit. Gaussian, binomial and Poisson projections give no warnings.
  • gamm4::gamm4(): a Gaussian submodel gives the REML mgcv::gam() fit with s(g, bs = "re"). The traditional and the augmented-data form of a binomial submodel give the same fit.
  • ordinal::clmm(): without group-level variation, the fit equals the MASS::polr() fit. Observations with weight 0 do not change the fit, and integer weights give the same fit as duplicated observations.
  • mclogit::mblogit(): without group-level variation, the fit equals the nnet::multinom() fit.
  • filter_mssgs_warns() removes the warnings about non-integer Poisson responses and keeps other messages.

  The version gate skip_if_lme4_rounds() skips the tests that need non-integer binomial or Poisson responses in glmer() with lme4 < 2.1-1, because these versions fit rounded data. lme4 2.1-1 (lme4's master with use continuous extension for binomial/Poisson aic() (GH #180, #682, #… lme4/lme4#1012) is not on CRAN yet. If the released version has another number, I will change the gate. Inside test_that(), testthat handles messages and warnings before R prints them, so capt_mssgs_warns() captures nothing there. The test helper with_r_printing() prints messages and warnings as R does with options(warn = 1). Thus the tests of the warning filters see the lines that capt_mssgs_warns() captures outside of tests. The file replaces test_glmer_nonint.R.

• R/divergence_minimizers.R:

  • The filtering of the captured submodel messages in divmin() moves into the new helper filter_mssgs_warns(). The helper also filters two warning patterns: "non-integer response values in a poisson model" (added by use continuous extension for binomial/Poisson aic() (GH #180, #682, #… lme4/lme4#1012) and the dpois() warning "non-integer x = ..." (for example, from the glm() start fit of glmmPQL()). The responses of a projection are in general non-integer. Without the filter, projpred passes these warnings on, and check_conv() reports every such submodel fit as possibly not converged.
  • fit_glmer_callback() calls glmmPQL() with verbose = FALSE unless the user sets verbose. Before this change, the "iteration" messages of glmmPQL() gave a warning for every projection.
  • check_conv_s() treats glmmPQL fits as converged. glmmPQL() stores no convergence information, and nlme::lme() throws an error if it does not converge. Before this change, every such fit gave "Unrecognized submodel fit. Please notify the package maintainer." and a non-convergence warning.

• NEWS.md entries.

Not changed: glmmPQL() with more than one group term fails ("object 'g' not found"). glmmPQL() gets the variables of a pdBlocked random structure from all.vars(formula(x)), and formula() of a pdBlocked object is a list. Observations with weight 0 make glmmPQL() fail and change the fit of gamm4() and mblogit(). #580 removes such observations before the fit. Tests for these cases can follow #580.

Tests

The new file passes with lme4 2.0-6, 2.1-0 and the development version 2.1-1 (lme4's master f86b09b7, with lme4/lme4#1012). With lme4 < 2.1-1, the version gate skips 5 of its 14 tests; without the gate, these 5 tests fail. Without the PQL changes, the PQL warning test fails ("iteration 1"). Without the two Poisson patterns in filter_mssgs_warns(), the filter test, the Poisson weight test and the PQL warning test fail.

With NOT_CRAN=true, the full suite has 267 tests. With lme4 2.0-6, it gives 0 failures and the same 3 warnings as master. With lme4 2.1-1, it gives 0 failures. With lme4 2.1-0, 4 tests in test_augdat.R fail; #577 fixes them. In CRAN mode (installed package, NOT_CRAN=false), 82 tests run with lme4 2.1-0 and 87 with lme4 2.1-1. All pass in about 60 seconds. The new file takes 1.5 and 2.1 seconds. Tested with R 4.6.1, brms 2.23.0, rstanarm 2.32.2, gamm4 0.2-7, mgcv 1.9-4, ordinal 2025.12-29, mclogit 0.9.15, MASS 7.3-65, nlme 3.1-169, on CSC Roihu (Linux, AMD EPYC).

R-hub (R-devel r90763 and r90766, lme4 2.1-0, CRAN mode) gives Status: OK with 0 test failures on 5 platforms: Windows, macOS arm64, Linux without long doubles (nold), Linux with ATLAS, and Linux with MKL. The 5 tests that need lme4 >= 2.1-1 skip there. The tests take 53 to 95 seconds on these platforms. R-hub run (tarball of c1c941ad, same tree as 71be2e8, plus Remotes: stan-dev/cmdstanr for the dependency installer): https://github.com/r-hub2/preferred-boa-projpred/actions/runs/40467574117

Overlap with other PRs

• Leave the left-out observation out of the fold projections in validated LOO CV #580 and Faster projections onto multilevel and additive submodels #584 add functions directly after divmin() in R/divergence_minimizers.R, where this PR adds filter_mssgs_warns(). The merge gives a text conflict. Keep the functions of all three PRs.
• NEWS.md: Keep the data-dependent bases #573, Adapt the tests to lme4 2.1-0 #577 and Leave the left-out observation out of the fold projections in validated LOO CV #580 add entries at the same place. The merge gives a text conflict. Keep all entries.
• Trial merges (2026-10-27) with the local branches of Keep the data-dependent bases #573, Faster submodel projections: one model matrix and one Gaussian solve for all draws #574, Adapt the tests to lme4 2.1-0 #577 and Leave the left-out observation out of the fold projections in validated LOO CV #580-Faster projections onto multilevel and additive submodels #584 gave only the conflicts above. Not checked: Vectorize the draw-wise predictive variance of the Gaussian and Student-t #575, Cluster the draws in the coordinates of their low-dimensional span #576, Avoid repeated formula updates, term extraction and training-data model matrices #578, Gaussian forward search and validated LOO evaluation from cross-products of the model matrix #579.

With lme4 2.1-0 (the CRAN version that the GitHub checks install) and NOT_CRAN=true, the full suite has the 4 test_augdat.R failures that #577 fixes. The GitHub checks fail until #577 is merged.