Skip to content

Run Nx integration tests sequentially to prevent shared-sandbox race condition#380

Merged
rhoerr merged 1 commit into
mage-os:mainfrom
thebraziliandeveloper:fix/nx-integration-tests-sequential-execution
May 27, 2026
Merged

Run Nx integration tests sequentially to prevent shared-sandbox race condition#380
rhoerr merged 1 commit into
mage-os:mainfrom
thebraziliandeveloper:fix/nx-integration-tests-sequential-execution

Conversation

@thebraziliandeveloper
Copy link
Copy Markdown
Contributor

Problem

When Nx runs multiple affected projects in parallel (default --parallel=3), each project's PHPUnit bootstrap runs concurrently inside the same Warden php-fpm container, sharing the same bind-mounted filesystem. This produces two classes of race condition that crash the integration test bootstrap before any tests execute:

1. deployTestModules.php conflicts

Both bootstraps call deployTestModules.php simultaneously. Each process removes and re-copies to app/code/Magento/TestModule*/ at the same time — one process deletes directories the other just created, or two mkdir() calls collide on the same path, triggering a "File exists" PHP warning that the integration test error handler converts into a fatal PHPUnit\Framework\Exception.

2. Shared sandbox directory conflict

Both bootstraps call setup:install via Magento\TestFramework\Application::install(), which creates a sandbox directory under:

dev/tests/integration/tmp/sandbox-0-<sha256(sha1(install-config))>/

Because both targets share the same phpunit.xml.dist and install-config-mysql.php, they compute an identical sandbox path. The second process to call mkdir() on that path fails with "File exists", again converted to a fatal bootstrap exception.

Fix

Add --parallel=1 to the nx affected command so test targets execute sequentially:

-nx affected --target=test:integration --head=HEAD --base=remotes/origin/${{ inputs.pr_base }}
+nx affected --target=test:integration --head=HEAD --base=remotes/origin/${{ inputs.pr_base }} --parallel=1

Trade-off

Sequential execution increases wall-clock time when multiple packages are affected simultaneously. However:

  • Most PRs touch 1–2 packages, so the practical impact is small.
  • The Warden stack (DB, Redis, RabbitMQ, OpenSearch) is a shared resource anyway, so true parallelism offers limited speed benefit.
  • The tests were failing entirely without this change — reliability over speed.

A longer-term fix would give each test target a distinct install config (different DB name), producing unique sandbox paths and eliminating the shared-state problem while restoring parallelism.

Related

Follows #379 (cache key fix, already merged).

🤖 Generated with Claude Code

When Nx runs multiple affected projects in parallel (default --parallel=3),
each project's PHPUnit bootstrap runs concurrently inside the same Warden
php-fpm container, sharing the same filesystem. This causes two classes of
race condition:

1. Both bootstraps call deployTestModules.php simultaneously, each removing
   and re-copying to app/code/Magento/TestModule*/ at the same time.

2. Both bootstraps call setup:install, which creates a sandbox directory
   under dev/tests/integration/tmp/ keyed by a SHA-256 hash of the install
   config. Since both targets share the same phpunit.xml.dist and
   install-config-mysql.php, they produce an identical sandbox path. The
   second process to arrive fails with "mkdir(): File exists", which the
   PHPUnit error handler converts to a fatal exception that crashes the
   bootstrap before any tests run.

Setting --parallel=1 makes Nx execute test targets sequentially, eliminating
the shared-state conflicts. The tradeoff is longer wall-clock time when
multiple packages are affected, but the tests were failing entirely without
this change so reliability wins over speed.
Copy link
Copy Markdown
Contributor

@rhoerr rhoerr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Seems plausible and safe. Merging to try it on mage-os/mageos-magento2#213 .

@rhoerr rhoerr merged commit 84d1462 into mage-os:main May 27, 2026
1 check passed
thebraziliandeveloper added a commit to thebraziliandeveloper/mageos-magento2 that referenced this pull request May 27, 2026
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants