Skip to content

fix(phpunit): report empty discovery as an empty result - #2570

Merged
chubes4 merged 1 commit into
mainfrom
fix/phpunit-discovery-empty-result
Oct 21, 2026
Merged

chubes4 merged 1 commit into
mainfrom
fix/phpunit-discovery-empty-result

Conversation

@chubes4

@chubes4 chubes4 commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

Part of Extra-Chill/homeboy#16202

Problem

wordpress.phpunit discovery-only=1 could not answer "no PHPUnit files" for a component whose tests are all standalone PHP smokes (for example Extra-Chill/extrachill-community: 28 files under tests/, none *Test.php, no phpunit.xml). Two gates turned that valid answer into a crash:

  1. The discovery-only PHP block threw RuntimeException: PHPUnit discovery found no test files.
  2. readPluginPhpunitDiscoveryResult rejected files.length === 0, so the command then failed with discovery-only completed without a valid discovery result.

Homeboy's WordPress extension builds its test inventory from this discovery plus the component's standalone PHP tests. Its contract is "empty PHPUnit discovery is valid when the standalone list is non-empty; both empty is refused". Because discovery crashed, every standalone-only component's review test evidence came back invalid_evidence: internal runtime inventory producer failed … WP Codebox canonical PHPUnit discovery failed.

Change

  • In discovery-only mode an empty set is reported as files: [] with the existing NO_TEST_FILES log line, instead of throwing.
  • The discovery reader accepts an empty files array. Every element must still be a non-empty string.
  • Unchanged: execution mode (discover_tests before running PHPUnit) still refuses to run nothing, and an unreadable test root still fails.

Verification

  • New test: discovery-only reports an empty test root as an empty result.
    • It runs the generated discovery-only PHP against a tests/ directory that holds only a *-smoke.php file, and asserts exit 0 with files: [].
    • It then feeds that result through runPhpunitCommand.
    • Mutation check: restoring the throw makes the test fail.
  • Related suites pass: phpunit-discovery-only (9), phpunit-changed-test-filter, phpunit-project-autoload, phpunit-result-paths, phpunit-runtime-failure-diagnostics, phpunit-runtime-rejection, phpunit-structured-evidence and wordpress-runtime-discovery-contracts.
  • End to end: homeboy review test extrachill-community --placement local with this build (HOMEBOY_WP_CODEBOX_BIN) and the paired extension change. Discovery returns files: [], the inventory is produced, and test evidence is status: complete. The run's only failures are the component's 4 genuinely failing smokes. Before this change the same run reported PHPUnit discovery found no test files.

AI disclosure: root cause, fix and tests by Claude Opus 5.5 (Pi coding agent) at Chris Huber's direction; reproduced and verified locally.

Discovery-only mode threw "PHPUnit discovery found no test files" and
the discovery reader rejected files: [], so a component whose suite is
entirely outside PHPUnit (standalone PHP smokes) could not get a
discovery answer at all. Discovery now reports what exists; consumers
decide whether zero PHPUnit files is acceptable. Execution mode still
refuses to run nothing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant