Record and show why items were deselected, without patching nodes - #15035
RonnyPfannschmidt wants to merge 3 commits into
Conversation
pytest_deselected carries no reason and cannot grow one: plugins call the hook, so a new argument would be one no existing caller passes, and pluggy cannot evolve the arguments of a hook call. Side-channel the reason through the config stash for the duration of the call instead, and record one at each of pytest's own deselection sites: -k, -m, --deselect, --lf and --stepwise. Nothing here is public - a plugin can neither supply a reason nor read one - because there is no point in exposing a channel the hook itself cannot carry. Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
Closes pytest-dev#13617. "collected 4 items / 4 deselected / 0 selected" says nothing about which items went away or why. At -v and above, write a deselected section that groups the items by the reason recorded for the pytest_deselected call that removed them. Items deselected by a plugin are listed under a placeholder, since no reason can reach us. Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
ef9de34 to
392c316
Compare
Stash.replaced(key, value) sets the key for the duration of a block and puts back what was there before, or removes the key again if it held nothing. Restoring absence is the part that is easy to get wrong by hand, and nesting then falls out for free. Use it for the deselection reason, which is the one place in the tree that wants a stash value for exactly one call. Co-Authored-By: Claude Opus 5 (1M context) via Claude Code <noreply@anthropic.com>
|
@bluetech i beleive theres bikeshedding needed on the new context manager, i also beleive we have a number of places where we should probably use one even if it needs code flow changes |
bluetech
left a comment
There was a problem hiding this comment.
The obvious spelling is pytest_deselected(items, reason). It is not available:
pytest_deselected is a hook third-party plugins call — calling it from a
pytest_collection_modifyitems implementation is part of the documented contract — and
pluggy cannot evolve the arguments of a hook call (pytest-dev/pluggy#170, where this
exact hook is named; pytest-dev/pluggy#361 for the call-control idea that would fix it).
Adding reason to the hookspec leaves every existing caller passing none, and a caller
that cannot pass one is indistinguishable from one that has nothing to say — so the
argument could never become required either.
Looking at pytest-dev/pluggy#170 and skimming the discussion there and in previous issues, I still didn't quite follow why the hookspec kwarg=default solution was rejected. That solution seems obvious to me and should be straightforward to implement. I can try and take a closer look at it and send a pluggy PR for this feature.
The side-channel seems a bit complex, and also doesn't allow plugins to participate, so the pluggy route seems better to me if we can do it.
-v and above gets a deselected section, grouped by reason:
This will be very noisy for large test suites. For example, try pytest -k foo -v in pytest's own test suite, which is a very common thing to do. It might even be too verbose for -vv. I think the UI for this should be reconsidered.
Supersedes #13618, closes #13617.
#13618 recorded the reason by patching a
_deselected_reasonattribute onto everyNode. This does the same job without touching the node: the reason travels next tothe
pytest_deselectedcall.Why a side channel
The obvious spelling is
pytest_deselected(items, reason). It is not available:pytest_deselectedis a hook third-party plugins call — calling it from apytest_collection_modifyitemsimplementation is part of the documented contract — andpluggy cannot evolve the arguments of a hook call (pytest-dev/pluggy#170, where this
exact hook is named; pytest-dev/pluggy#361 for the call-control idea that would fix it).
Adding
reasonto the hookspec leaves every existing caller passing none, and a callerthat cannot pass one is indistinguishable from one that has nothing to say — so the
argument could never become required either.
So
_pytest/deselect.pystashes the reason on the config for the duration of the hookcall and pytest reads it back in its own reporting. That is a workaround, and it is
deliberately private: a plugin can neither supply a reason nor read one. Exposing
a channel the hook itself cannot carry would only entrench the workaround. Follow-up for
making it a real API: #15036
What it looks like
-vand above gets adeselectedsection, grouped by reason:Reasons are recorded at all of pytest's own deselection sites:
-k,-m,--deselect,--lf(both the "passed last run" and the--last-failed-no-failures=nonepaths) and--stepwise. Items deselected by a pluginare listed under
deselected by a plugin, no reason recorded— still an answer to"which items went away", which is what the issue's later reporter was missing.
Default output is unchanged.
Stash.replaced()The side channel needs a stash value set for exactly the length of one call, and
restoring absence is the part that is easy to get wrong by hand. So this also adds
Stash.replaced(key, value):It restores the previous value, or removes the key again if there was none, and nests.
It is in this PR rather than its own because deselection is the only block-scoped stash
value in the tree -- every other stash lifetime (logging, faulthandler, unraisable,
mark) spans separate hook calls, which a context manager cannot express, so a standalone
PR would be adding an API with no caller.
Notes for review
deselect_items()saves and restores the previous reason, so apytest_deselectedimplementation that deselects further items does not clobber the outer one; covered by
a test.
testing/test_stepwise.pychanged: it asserted thedeselected test's name was absent from
-voutput, which is now false by design. Itnow asserts the test did not run.
pre-commit run -aclean.🤖 Generated with Claude Code