Summary
RDKit is currently pinned to at least 5 different versions across this repo, split between the Docker build path and the local/conda dev-environment files:
Docker images (all built FROM informaticsmatters/vs-rdkit-base:latest, which pip-installs RDKit once and is shared by Dockerfile-fns, Dockerfile-mordred, Dockerfile-prep, Dockerfile-oddt, Dockerfile-moldb):
Dockerfile-rdkit-base: rdkit==2023.3.2 (pip)
Conda environment files (used for local development/testing per USER_GUIDE.md / tests/README.md, not the production image builds):
environment-moldb.yaml: rdkit=2021.03.5
environment-im-fns.yaml: rdkit=2021.03.5
environment-im-mordred.yaml: rdkit=2023.09.5
environment-im-vs-prep.yaml: rdkit=2022.09.4
environment-im-oddt.yaml: rdkit=2023.03.3
So a developer testing locally via conda for, say, mordred is validating against RDKit 2023.09.5, while the actual shipped container (vs-rdkit-base) runs 2023.3.2 — a different minor release with potentially different behaviour/output. None of the five versions above match each other.
Also relevant: rdkit==2023.3.2 predates NumPy 2.0, which recently caused real test failures (AttributeError: _ARRAY_API not found) once NumPy resolved to 2.x at build time — fixed for now in #19 by pinning numpy<2, but that's a workaround, not a fix to the version drift itself.
Proposal
- Pick one target RDKit version (ideally a recent release with native NumPy 2 support, e.g. 2024.03+) and standardize on it everywhere:
Dockerfile-rdkit-base and all five environment-*.yaml conda files.
- Re-validate the full
jote test suite against the chosen version (several jobs — im-mordred, im-virtual-screening/rdkit, moldb, silicos-it, fns — depend on vs-rdkit-base and would need re-testing for behavioural drift).
- Consider whether the conda environment files should be regenerated/pinned from the same source as the Docker image (or deprecated in favour of testing directly against the built image) to prevent this kind of drift recurring.
How this was found
Found while running a full (non-dry-run) jote pre-refactor health check across all job submodules in the squonk2-jobs umbrella repo, and while fixing the NumPy 2 ABI break in #19.
Summary
RDKit is currently pinned to at least 5 different versions across this repo, split between the Docker build path and the local/conda dev-environment files:
Docker images (all built
FROM informaticsmatters/vs-rdkit-base:latest, which pip-installs RDKit once and is shared byDockerfile-fns,Dockerfile-mordred,Dockerfile-prep,Dockerfile-oddt,Dockerfile-moldb):Dockerfile-rdkit-base:rdkit==2023.3.2(pip)Conda environment files (used for local development/testing per
USER_GUIDE.md/tests/README.md, not the production image builds):environment-moldb.yaml:rdkit=2021.03.5environment-im-fns.yaml:rdkit=2021.03.5environment-im-mordred.yaml:rdkit=2023.09.5environment-im-vs-prep.yaml:rdkit=2022.09.4environment-im-oddt.yaml:rdkit=2023.03.3So a developer testing locally via conda for, say,
mordredis validating against RDKit 2023.09.5, while the actual shipped container (vs-rdkit-base) runs 2023.3.2 — a different minor release with potentially different behaviour/output. None of the five versions above match each other.Also relevant:
rdkit==2023.3.2predates NumPy 2.0, which recently caused real test failures (AttributeError: _ARRAY_API not found) once NumPy resolved to 2.x at build time — fixed for now in #19 by pinningnumpy<2, but that's a workaround, not a fix to the version drift itself.Proposal
Dockerfile-rdkit-baseand all fiveenvironment-*.yamlconda files.jotetest suite against the chosen version (several jobs —im-mordred,im-virtual-screening/rdkit,moldb,silicos-it,fns— depend onvs-rdkit-baseand would need re-testing for behavioural drift).How this was found
Found while running a full (non-dry-run)
jotepre-refactor health check across all job submodules in thesquonk2-jobsumbrella repo, and while fixing the NumPy 2 ABI break in #19.