Skip to content

fixtures: say where a fixture is defined, when it is missing and when it is doubled - #15073

Draft
RonnyPfannschmidt wants to merge 2 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:feature/fixture-visibility-diagnostics
Draft

RonnyPfannschmidt wants to merge 2 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:feature/fixture-visibility-diagnostics

Conversation

@RonnyPfannschmidt

@RonnyPfannschmidt RonnyPfannschmidt commented Sep 21, 2026

Copy link
Copy Markdown
Member

Written by Claude Opus 5 (1M context) via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

Two long-open fixture-diagnostics issues that turn out to be the same complaint from opposite ends: pytest knows where a fixture name is defined and does not say so.

Draft: both are implemented and green. Draft because #1511 turns on a new warning for a pattern that is widespread in the wild, and that is a call for the team, not for me — see the question at the end.


#10151 — hint at out-of-scope definitions

A fixture moved into a conftest one directory over stays registered — it is just not visible where it is being requested. The error said only that the name was not found, then listed the fixtures that were available. Neither half mentions the definition sitting in the tree, so the reader goes looking for a name that is already in front of them.

mypkg/
  tests/
    conftest.py     # myfixture lives here now
    test_foo.py     # works
  utils/tests/
    test_bar.py     # "fixture 'myfixture' not found"

One hint line, between the failure and the available list:

E       fixture 'myfixture' not found
>       hint: 'myfixture' is defined in mypkg/tests/conftest.py:4, but not visible here
>       available fixtures: capfd, capfdbinary, caplog, ...
>       use 'pytest --fixtures [testpath]' for help on them.

Past five distinct locations the paths stop being a hint and start being a wall, so the tail collapses into a count:

>       hint: 'shared' is defined in 7 places, none visible here: a/conftest.py:4, b/conftest.py:4, c/conftest.py:4, d/conftest.py:4, e/conftest.py:4, and 2 more

Locations are rendered with the existing _pretty_fixture_path, so pytest's own builtins show as .../_pytest/... rather than a site-packages path.

Why now

This has been open since 2022 because answering "where else is this name defined?" used to mean scanning the whole definition list for that name, which is not something to do on an error path in a large suite. After #14984 the fixturedefs for a name are a dict keyed by node, so the enumeration is a lookup. The new FixtureManager._get_all_fixture_defs_for_name() sits next to the existing _get_all_fixture_defs* helpers and is three lines.

Known limitation

The hint only sees conftests that the run actually loaded. Collecting a single file does not load its siblings, so running just mypkg/utils/tests/test_bar.py reports nothing. A whole-suite run — which is how the confusion arises in the first place — has them all. This is inherent to lazy conftest loading and is called out in the commit message.


#1511 — warn on imported fixtures

Importing a fixture to reuse it binds it in the importing module too. If the module it came from is itself a plugin or a collected module, both are fixture holders, parsefactories runs over both, and the same function is registered twice under two different visibilities. A session-scoped fixture registered twice runs twice, and neither file shows why. Open since 2016.

New PytestImportedFixtureWarning, raised on the second registration and pointed at the line that brought the fixture in:

b/conftest.py:1: PytestImportedFixtureWarning: fixture 'shared', defined in 'helpers', is registered twice: by 'a.conftest' and by 'b.conftest'.
Importing a fixture registers another copy of it, so it can run more than once and shadow other definitions of the same name.
Drop the import and reach it through a conftest.py, or list 'helpers' in 'pytest_plugins'.
  from helpers import shared

The trigger is the duplicate, not the import

The first version of this warned on the import itself, and the plugins CI job rejected it: pytest_django/plugin.py does from pytest_django.fixtures import _django_db_helper, and pytest_django.fixtures is never itself registered. One holder, one registration, no bug — a false positive, and a whole class of them.

So the check is now whether two module holders register the same function object. An import whose origin module is never a holder registers the fixture exactly once and behaves exactly as if it were defined locally, so it is silent. That distinction is what separates the bug from the shapes that merely resemble it:

shape warns why
plugin keeps fixtures in a sibling module of its own (pytest-django) no the sibling is never a holder — one registration
from helpers import shared, helpers never collected or a plugin no one registration; behaves as if defined locally
class inherits fixtures from a base in another module no same function once per subclass, by design — only module holders are tracked at all
fixture factory defined elsewhere no a fresh function per call, so two calls are two fixtures
two conftests both from helpers import shared yes two holders, two registrations
import from a module that is also collected or a plugin yes same

Each row has a test.

Anchoring

The warning is anchored at the second module's file, not at a line in it. Which line bound the name is not recorded anywhere — it is gone once the module is imported — and recovering it would mean parsing the source at warning time, which is not worth it for a line the reader can find by looking at the top of the file. The message names both registering modules and the defining one, which is what there is to act on.

Verification against real plugins

tox -e plugins is what caught the false positive, so it is also the evidence: the integration run now passes with all 14 plugins loaded, pytest-django among them. (bdd_wallet.py and pytest_rerunfailures_integration.py fail in that directory on main too, for unrelated reasons — the former is the _arg2node2fixturedefs breakage already commented out in tox.ini — and neither is part of the CI command.)

Bikeshedding, deliberately not done here

The #10151 sketch also reworded the following line to available fixtures for 'mypkg/utils/tests/test_bar.py':. It is independent of the hint, it would touch every test matching available fixtures:, and the exact wording is worth an argument on its own. Flagging it as a bikeshed rather than smuggling it in; happy to do it in a follow-up once someone picks a colour.


Checks

Full suite 4637 passed, 46 skipped, 17 xfailed, 4 xpassed under -n auto; tox -e plugins green; pre-commit run -a clean.

Fifteen tests added: five in TestFillFixtures for the hint, and ten in a new TestImportedFixtureWarning — one per row of the table above, plus a definition registered after the module that imported it, a plugin module built at runtime with no __file__, and what the message looks like once -W error strips the location off it.

Two commits, each standalone and each green on its own.

testing/test_debugging.py::TestPDB::test_pdb_interaction_exception fails intermittently under -n auto, on this branch and on an unmodified main alike — pre-existing, checked in a clean worktree, unrelated to this change.

The question for reviewers

PytestImportedFixtureWarning fires by default. It now fires only on genuine duplicate registration, which is narrower than the first cut, but suites that do this will still be noisy on first upgrade — which is the point of the issue, though the noise lands on people who did not ask for it. Options, in the order I would rank them:

  1. Ship it on by default, as here. It is a real duplicate-registration bug every time it fires.
  2. Make it a PytestDeprecationWarning so the existing deprecation machinery and timeline apply.
  3. Put it behind an opt-in flag or ini setting for a release, then flip the default.

Happy to do any of the three; this is a policy call rather than a technical one, which is why the PR is a draft.

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Sep 21, 2026
@RonnyPfannschmidt
RonnyPfannschmidt force-pushed the feature/fixture-visibility-diagnostics branch 3 times, most recently from f8dfb55 to 065014c Compare September 21, 2026 18:46
Moving a fixture into a conftest one directory over leaves the name
registered but out of scope, and the error said only that it was "not
found" plus a list of the fixtures that *were* available. Neither half
mentions the definition sitting in the tree, so the reader goes looking
for a name that is already in front of them.

List those definitions. The fixturedefs registered under a name are now
a dict keyed by node (pytest-dev#14984), so enumerating them regardless of
visibility is a lookup rather than a scan -- which is what makes this
cheap enough to do on the error path.

Past five distinct locations the paths stop being a hint and start being
a wall, so collapse the tail into a count.

The hint only sees conftests that this run actually loaded. Collecting a
single file does not load its siblings, so the case that prompted the
issue reports nothing when run that narrowly; a whole-suite run, which
is how the confusion arises in the first place, has them all.

Fixes pytest-dev#10151.

Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
@RonnyPfannschmidt
RonnyPfannschmidt force-pushed the feature/fixture-visibility-diagnostics branch from 065014c to 4e3c9a7 Compare September 21, 2026 18:48
Importing a fixture to reuse it binds it in the importing module too. If
the module it came from is itself a plugin or a collected module, both
are fixture holders, parsefactories runs over both, and the same function
is registered twice under two different visibilities. A session-scoped
fixture registered twice runs twice, and neither file shows why -- the
import looks like any other import. This is pytest-dev#1511, open since 2016.

Warn on the second registration, with PytestImportedFixtureWarning.

The trigger is the duplicate, not the import. An import whose origin
module is never itself a holder registers the fixture exactly once and
behaves exactly as if it were defined locally, so it is not warned about.
That distinction is what separates the bug from the shapes that merely
look like it:

- a plugin keeping its fixtures in a sibling module of its own and
  importing them into the module it registers, which is pytest-django's
  layout and registers each fixture once;
- a class inheriting fixtures from a base in another module, which
  registers the same function once per subclass, by design -- so only
  module holders are tracked at all;
- a fixture factory defined elsewhere, which returns a fresh function per
  call, so two calls are two fixtures rather than one registered twice.

The warning is anchored at the second module's file, not at a line in it.
Which line bound the name is not recorded anywhere -- it is gone once the
module is imported -- and recovering it would mean parsing the source at
warning time. The message names both modules and the defining one, which
is what the reader acts on; the import is at the top of the file it
names.

Fixes pytest-dev#1511.

Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
@RonnyPfannschmidt
RonnyPfannschmidt force-pushed the feature/fixture-visibility-diagnostics branch from 4e3c9a7 to 26eec22 Compare September 21, 2026 19:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

1 participant