Skip to content

Standardize and update the RDKit version used across the repo #20

Description

@tdudgeon

Summary

RDKit is currently pinned to at least 5 different versions across this repo, split between the Docker build path and the local/conda dev-environment files:

Docker images (all built FROM informaticsmatters/vs-rdkit-base:latest, which pip-installs RDKit once and is shared by Dockerfile-fns, Dockerfile-mordred, Dockerfile-prep, Dockerfile-oddt, Dockerfile-moldb):

  • Dockerfile-rdkit-base: rdkit==2023.3.2 (pip)

Conda environment files (used for local development/testing per USER_GUIDE.md / tests/README.md, not the production image builds):

  • environment-moldb.yaml: rdkit=2021.03.5
  • environment-im-fns.yaml: rdkit=2021.03.5
  • environment-im-mordred.yaml: rdkit=2023.09.5
  • environment-im-vs-prep.yaml: rdkit=2022.09.4
  • environment-im-oddt.yaml: rdkit=2023.03.3

So a developer testing locally via conda for, say, mordred is validating against RDKit 2023.09.5, while the actual shipped container (vs-rdkit-base) runs 2023.3.2 — a different minor release with potentially different behaviour/output. None of the five versions above match each other.

Also relevant: rdkit==2023.3.2 predates NumPy 2.0, which recently caused real test failures (AttributeError: _ARRAY_API not found) once NumPy resolved to 2.x at build time — fixed for now in #19 by pinning numpy<2, but that's a workaround, not a fix to the version drift itself.

Proposal

  • Pick one target RDKit version (ideally a recent release with native NumPy 2 support, e.g. 2024.03+) and standardize on it everywhere: Dockerfile-rdkit-base and all five environment-*.yaml conda files.
  • Re-validate the full jote test suite against the chosen version (several jobs — im-mordred, im-virtual-screening/rdkit, moldb, silicos-it, fns — depend on vs-rdkit-base and would need re-testing for behavioural drift).
  • Consider whether the conda environment files should be regenerated/pinned from the same source as the Docker image (or deprecated in favour of testing directly against the built image) to prevent this kind of drift recurring.

How this was found

Found while running a full (non-dry-run) jote pre-refactor health check across all job submodules in the squonk2-jobs umbrella repo, and while fixing the NumPy 2 ABI break in #19.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions