Skip to content

Tests always skipped when dependency version is present #318

Description

@KostaIlic2

__doctest_requires__ causes tests to be skipped whenever a dependency version requirement is present, even if the dependency is present and meets the requirement.

The code was behaving correctly in 1.5.0. The bug was present in 1.6.0. The bug is still present in 1.7.0.

To reproduce, save code fragments shown below as demo.py and pyproject.toml into an empty project root directory, then run the following commands from that directory:

  1. Create virtual environment: python -m venv .venv
  2. Install dependencies: .\.venv\Scripts\pip.exe install -e .
  3. Show NumPy version: .\.venv\Scripts\pip.exe show numpy
  4. Read the output to verify NumPy >= 2.0 is installed.
  5. Run doctest: .\.venv\Scripts\python.exe -m pytest demo.py --doctest-modules -v
  6. Observe the output depending on the version of pytest-doctestplus:
    1.5.0: demo.py::demo.my_function PASSED
    1.6.0: demo.py::demo.my_function SKIPPED (could not import 'numpy>=2.0': No module named 'numpy>=2')
    1.7.0: demo.py::demo.my_function SKIPPED (could not import 'numpy>=2.0': No module named 'numpy>=2')
  7. Upgrade pytest-doctestplus to 1.6.0 and run the test again, following the steps
    mentioned above.
  8. Upgrade pytest-doctestplus to 1.7.0 and run the test again, following the steps
    mentioned above.
#demo.py

__doctest_requires__ = {("my_function",): ["numpy>=2.0"]}

def my_function():
    """Simple function to demonstrate the bug.
    >>> import numpy as np
    >>> np.array([1, 2, 3])
    array([1, 2, 3])
    """
    pass
#pyproject.toml

[project]
name = "pytest-doctestplus-bug-demo"
version = "0.1.0"
description = "Demonstrates pytest-doctestplus 1.7.0 bug with version constraints"
requires-python = ">=3.9"
dependencies = [
    "numpy>=2.0",
    "pytest-doctestplus==1.5.0",
    "pytest>=7.0",
]

[build-system]
requires = ["setuptools>=45", "wheel"]
build-backend = "setuptools.build_meta"

[tool.pytest.ini_options]
addopts = "--doctest-modules --doctest-plus"

Activity

  1. KostaIlic2 commented on Jan 20, 2026

    @KostaIlic2
    ContributorAuthor

    I have looked at the problem with help of LLM tools. I have a potential fix and a regression test improvement that covers the scenarios from the bug. Would you like me to post a PR?

  2. pllim commented on Jan 20, 2026

    @pllim
    Contributor
  3. pllim commented on Jan 20, 2026

    @pllim
    Contributor

    Would you like me to post a PR?

    Perhaps run a test locally first to make sure LLM suggestion actually works? If unit tests pass, then PR is fine. Thanks.

  4. pllim commented on Jan 20, 2026

    @pllim
    Contributor

    Did you try to bisect it first?

  5. bsipocz commented on Jan 20, 2026

    @bsipocz
    Member

    Before you unleash LLM on this, it would be really useful to see an actually failing. Here are some guidance of how to write such examples; ideally they are just a test function (I certainly will not download attached zip files): https://matthewrocklin.com/minimal-bug-reports/

  6. KostaIlic2 commented on Jan 20, 2026

    @KostaIlic2
    ContributorAuthor

    Dear @bsipocz and @pllim, Thank you for your prompt responses and clear direction.

    I rewrote the report per your request to avoid a zip file.

    I mentioned LLM to be upfront about it. I reviewed the changes made by the LLM, made adjustments myself, and then had another level of coworker review. Changes to the core file are about 10 lines of code, and changes to the test are about 30 lines of code.

    I did not track down the PR that introduced the bug.

  7. KostaIlic2 commented on Jan 21, 2026

    @KostaIlic2
    ContributorAuthor

    To follow up on my earlier response, I have run updated test locally. Without the plugin.py code change, the test fails. With the change, the test passes.

    I bisected and found the bug was introduced in PR #312 (commit 3188e5b).

    I have created a draft PR #319 .

    Do you have any expectations from contributors beyond updates to the code, the tests, and the changelog?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions