Skip to content

BUG: test failures with windows+pytest-dev+py3.14 #305

Description

@bsipocz

It feels like something upstream, but I cannot really debug it (it only shows on windows). Maybe @pllim you can have a look at it next week, and see if we need to report it upstream?

https://github.com/scientific-python/pytest-doctestplus/actions/runs/18484974305/job/53059800758#step:5:805

py314-test-pytestdev: commands[1] D:\a\pytest-doctestplus\pytest-doctestplus\.tmp\py314-test-pytestdev> pytest D:\a\pytest-doctestplus\pytest-doctestplus/tests --ignore=D:\a\pytest-doctestplus\pytest-doctestplus/tests/docs/skip_some_remote_data.rst --doctest-plus --doctest-rst --remote-data
============================= test session starts =============================
platform win32 -- Python 3.14.0, pytest-8.5.0.dev201+gc97a40140, pluggy-1.6.0
cachedir: .tox\py314-test-pytestdev\.pytest_cache
rootdir: D:\a\pytest-doctestplus\pytest-doctestplus
configfile: setup.cfg
plugins: doctestplus-1.4.1.dev37+g2add52568, remotedata-0.4.1
collected 75 items

..\..\tests\docs\skip_all.rst s                                          [  1%]
..\..\tests\docs\skip_some.rst .                                         [  2%]
..\..\tests\python\doctests.py ...                                       [  6%]
..\..\tests\test_doctestplus.py ...................x.................... [ 60%]
............                                                             [ 76%]
..\..\tests\test_encoding.py ........ssssssss                            [ 97%]
..\..\tests\test_utils.py ..                                             [100%]

================== 65 passed, 9 skipped, 1 xfailed in 15.51s ==================
py314-test-pytestdev: exit 0 (16.21 seconds) D:\a\pytest-doctestplus\pytest-doctestplus\.tmp\py314-test-pytestdev> pytest D:\a\pytest-doctestplus\pytest-doctestplus/tests --ignore=D:\a\pytest-doctestplus\pytest-doctestplus/tests/docs/skip_some_remote_data.rst --doctest-plus --doctest-rst --remote-data pid=7040
py314-test-pytestdev: commands[2] D:\a\pytest-doctestplus\pytest-doctestplus\.tmp\py314-test-pytestdev> pytest D:\a\pytest-doctestplus\pytest-doctestplus/tests
============================= test session starts =============================
platform win32 -- Python 3.14.0, pytest-8.5.0.dev201+gc97a40140, pluggy-1.6.0
cachedir: .tox\py314-test-pytestdev\.pytest_cache
rootdir: D:\a\pytest-doctestplus\pytest-doctestplus
configfile: setup.cfg
plugins: doctestplus-1.4.1.dev37+g2add52568, remotedata-0.4.1
collected 70 items

..\..\tests\test_doctestplus.py ...................x...F................ [ 57%]
............                                                             [ 74%]
..\..\tests\test_encoding.py ........ssssssss                            [ 97%]
..\..\tests\test_utils.py ..                                             [100%]

================================== FAILURES ===================================
_____________________________ test_ignore_option ______________________________

testdir = <Testdir local('C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-unknown\\pytest-1\\test_ignore_option0')>

    def test_ignore_option(testdir):
        testdir.makepyfile(foo="""
            def f():
                '''
                >>> 1+1
                2
                '''
                pass
        """)
        testdir.makepyfile(bar="""
            def f():
                '''
                >>> 1+1
                2
                '''
                pass
        """)
        testdir.makefile('.rst', foo='>>> 1+1\n2')
    
config = <_pytest.config.Config object at 0x0000018A2069D090>

    def collect_unraisable(config: Config) -> None:
        pop_unraisable = config.stash[unraisable_exceptions].pop
        errors: list[pytest.PytestUnraisableExceptionWarning | RuntimeError] = []
        meta = None
        hook_error = None
        try:
            while True:
                try:
                    meta = pop_unraisable()
                except IndexError:
                    break
    
                if isinstance(meta, BaseException):
                    hook_error = RuntimeError("Failed to process unraisable exception")
                    hook_error.__cause__ = meta
                    errors.append(hook_error)
                    continue
    
                msg = meta.msg
                try:
>                   warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E                   pytest.PytestUnraisableExceptionWarning: Exception ignored while finalizing file <_io.FileIO name='D:\\a\\pytest-doctestplus\\pytest-doctestplus\\.tox\\py314-test-pytestdev\\Scripts\\pytest.EXE' mode='rb' closefd=True>: None

D:\a\pytest-doctestplus\pytest-doctestplus\.tox\py314-test-pytestdev\Lib\site-packages\_pytest\unraisableexception.py:67: PytestUnraisableExceptionWarning
=========================== short test summary info ===========================
ERROR bar.py::bar.f - pytest.PytestUnraisableExceptionWarning: Exception ignored while finalizing file <_io.FileIO name='D:\\a\\pytest-doctestplus\\pytest-doctestplus\\.tox\\py314-test-pytestdev\\Scripts\\pytest.EXE' mode='rb' closefd=True>: None
========================= 2 passed, 1 error in 0.12s ==========================
=========================== short test summary info ===========================
FAILED ..\..\tests\test_doctestplus.py::test_ignore_option - AssertionError: ([<TestReport 'foo.py::foo.f' when='call' outcome='passed'>, <TestReport 'foo.rst::foo.rst' when='call' outcome='passed'>], [], [<TestReport 'bar.py::bar.f' when='setup' outcome='failed'>])
assert {'failed': 1,... 'skipped': 0} == {'failed': 0,... 'skipped': 0}
  
  Omitting 1 identical items, use -vv to show
  Differing items:
  {'failed': 1} != {'failed': 0}
  {'passed': 2} != {'passed': 3}
  
  Full diff:
    {
  -     'failed': 0,
  ?               ^
  +     'failed': 1,
  ?               ^
  -     'passed': 3,
  ?               ^
  +     'passed': 2,
  ?               ^
        'skipped': 0,
    }
============= 1 failed, 60 passed, 8 skipped, 1 xfailed in 8.64s ==============
py314-test-pytestdev: exit 1 (9.20 seconds) D:\a\pytest-doctestplus\pytest-doctestplus\.tmp\py314-test-pytestdev> pytest D:\a\pytest-doctestplus\pytest-doctestplus/tests pid=6964
  py314-test-pytestdev: FAIL code 1 (70.95=setup[45.19]+cmd[0.35,16.21,9.20] seconds)
  evaluation failed :( (71.03 seconds)

Activity

  1. bsipocz commented on Oct 28, 2025

    @bsipocz
    MemberAuthor
  2. pllim commented on Oct 28, 2025

    @pllim
    Contributor

    Why do these things always happen around feature freeze? 😩

  3. pllim commented on Oct 29, 2025

    @pllim
    Contributor

    The new error is even crazier. It does not say from which test. Just dumps a traceback right in the middle of the test report that does not have failure (61 passed, 8 skipped, 1 xfailed).

  4. bsipocz commented on Oct 29, 2025

    @bsipocz
    MemberAuthor

    Why do these things always happen around feature freeze?

    😄

    But more seriously, this is not urgent here, I just kept tab on the failing crons and making sure we have some tracking issues open.

  5. pllim commented on Oct 29, 2025

    @pllim
    Contributor

    Appreciated it, @bsipocz ! I do not get those notifications so it is awesome that you are doing it.

  6. bsipocz commented on Oct 29, 2025

    @bsipocz
    MemberAuthor

    I have it on my list to set up the action to auto open issues 😄

  7. pllim commented on Nov 1, 2025

    @pllim
    Contributor

    The warning was ignored in #310 but we should keep this open because that PR did not solve the underlying issue. Best case scenario is we remove the ignore a year from now and it just magically works again...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions