Skip to content

Record and show why items were deselected, without patching nodes - #15035

Open
RonnyPfannschmidt wants to merge 3 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:feat/13617-deselect-reason-side-channel
Open

RonnyPfannschmidt wants to merge 3 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:feat/13617-deselect-reason-side-channel

Conversation

@RonnyPfannschmidt

@RonnyPfannschmidt RonnyPfannschmidt commented Sep 16, 2026

Copy link
Copy Markdown
Member

AI-authored. I prompted this exploration; the agent (Claude Opus 5 via Claude Code)
wrote the code, the tests and this description. I reviewed it and I am posting it, and
I will answer for it in review.

Supersedes #13618, closes #13617.

#13618 recorded the reason by patching a _deselected_reason attribute onto every
Node. This does the same job without touching the node: the reason travels next to
the pytest_deselected call.

Why a side channel

The obvious spelling is pytest_deselected(items, reason). It is not available:
pytest_deselected is a hook third-party plugins call — calling it from a
pytest_collection_modifyitems implementation is part of the documented contract — and
pluggy cannot evolve the arguments of a hook call (pytest-dev/pluggy#170, where this
exact hook is named; pytest-dev/pluggy#361 for the call-control idea that would fix it).
Adding reason to the hookspec leaves every existing caller passing none, and a caller
that cannot pass one is indistinguishable from one that has nothing to say — so the
argument could never become required either.

So _pytest/deselect.py stashes the reason on the config for the duration of the hook
call and pytest reads it back in its own reporting. That is a workaround, and it is
deliberately private: a plugin can neither supply a reason nor read one. Exposing
a channel the hook itself cannot carry would only entrench the workaround. Follow-up for
making it a real API: #15036

What it looks like

-v and above gets a deselected section, grouped by reason:

================================== deselected ==================================
-m 'not slow' did not match:
  test_it.py::test_slow

Reasons are recorded at all of pytest's own deselection sites: -k, -m,
--deselect, --lf (both the "passed last run" and the
--last-failed-no-failures=none paths) and --stepwise. Items deselected by a plugin
are listed under deselected by a plugin, no reason recorded — still an answer to
"which items went away", which is what the issue's later reporter was missing.

Default output is unchanged.

Stash.replaced()

The side channel needs a stash value set for exactly the length of one call, and
restoring absence is the part that is easy to get wrong by hand. So this also adds
Stash.replaced(key, value):

with config.stash.replaced(deselection_reason_key, reason):
    config.hook.pytest_deselected(items=items)

It restores the previous value, or removes the key again if there was none, and nests.
It is in this PR rather than its own because deselection is the only block-scoped stash
value in the tree -- every other stash lifetime (logging, faulthandler, unraisable,
mark) spans separate hook calls, which a context manager cannot express, so a standalone
PR would be adding an API with no caller.

Notes for review

  • deselect_items() saves and restores the previous reason, so a pytest_deselected
    implementation that deselects further items does not clobber the outer one; covered by
    a test.
  • One existing assertion in testing/test_stepwise.py changed: it asserted the
    deselected test's name was absent from -v output, which is now false by design. It
    now asserts the test did not run.
  • Full suite green, pre-commit run -a clean.

🤖 Generated with Claude Code

RonnyPfannschmidt and others added 2 commits September 16, 2026 09:16
pytest_deselected carries no reason and cannot grow one: plugins call the
hook, so a new argument would be one no existing caller passes, and pluggy
cannot evolve the arguments of a hook call.

Side-channel the reason through the config stash for the duration of the
call instead, and record one at each of pytest's own deselection sites:
-k, -m, --deselect, --lf and --stepwise.  Nothing here is public - a plugin
can neither supply a reason nor read one - because there is no point in
exposing a channel the hook itself cannot carry.

Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
Closes pytest-dev#13617.

"collected 4 items / 4 deselected / 0 selected" says nothing about which
items went away or why.  At -v and above, write a deselected section that
groups the items by the reason recorded for the pytest_deselected call that
removed them.  Items deselected by a plugin are listed under a placeholder,
since no reason can reach us.

Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
@RonnyPfannschmidt
RonnyPfannschmidt force-pushed the feat/13617-deselect-reason-side-channel branch from ef9de34 to 392c316 Compare September 16, 2026 07:16
Stash.replaced(key, value) sets the key for the duration of a block and puts
back what was there before, or removes the key again if it held nothing.
Restoring absence is the part that is easy to get wrong by hand, and nesting
then falls out for free.

Use it for the deselection reason, which is the one place in the tree that
wants a stash value for exactly one call.

Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
@RonnyPfannschmidt

Copy link
Copy Markdown
Member Author

@bluetech i beleive theres bikeshedding needed on the new context manager, i also beleive we have a number of places where we should probably use one even if it needs code flow changes

@bluetech bluetech left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The obvious spelling is pytest_deselected(items, reason). It is not available:
pytest_deselected is a hook third-party plugins call — calling it from a
pytest_collection_modifyitems implementation is part of the documented contract — and
pluggy cannot evolve the arguments of a hook call (pytest-dev/pluggy#170, where this
exact hook is named; pytest-dev/pluggy#361 for the call-control idea that would fix it).
Adding reason to the hookspec leaves every existing caller passing none, and a caller
that cannot pass one is indistinguishable from one that has nothing to say — so the
argument could never become required either.

Looking at pytest-dev/pluggy#170 and skimming the discussion there and in previous issues, I still didn't quite follow why the hookspec kwarg=default solution was rejected. That solution seems obvious to me and should be straightforward to implement. I can try and take a closer look at it and send a pluggy PR for this feature.

The side-channel seems a bit complex, and also doesn't allow plugins to participate, so the pluggy route seems better to me if we can do it.

-v and above gets a deselected section, grouped by reason:

This will be very noisy for large test suites. For example, try pytest -k foo -v in pytest's own test suite, which is a very common thing to do. It might even be too verbose for -vv. I think the UI for this should be reconsidered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

logging: pytest should allow users to know why a test was deselected

2 participants