Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ test run.
By default, no parallel execution takes place. Attributes are used to indicate which tests may run in parallel and how
they relate to other tests.

> [!TIP]
> This note describes how NUnit schedules parallel work. For practical questions, such as how to turn parallel execution
> on and what to do about state shared between tests, see
> [Running Tests in Parallel](../../writing-tests/parallel-tests.md).

## Platform Support

Parallel execution is supported by the NUnit framework on desktop .NET runtimes and .NET Standard 2.0. It is not
Expand Down Expand Up @@ -36,9 +41,11 @@ This is a `[Flags]` type enumeration used to specify which tests may run in para
it appears and any subordinate tests. The following values are available for use:

* `ParallelScope.Self` indicates that the test itself may be run in parallel with other tests. This is the default for
the `ParallelizableAttribute` and is the only value permitted on a test method.
the `ParallelizableAttribute` and is the only value permitted on a test method that is not parameterized.
* `ParallelScope.Children` indicates that the descendants of the test may be run in parallel with respect to one
another.
another. On a parameterized test method this is the only part of the scope that has an effect: the suite of test
cases always executes directly on the thread of its fixture, so `Self` is ignored there and `Children` and `All` are
equivalent. The individual cases are then dispatched to the parallel queue.
* `ParallelScope.Fixtures` indicates that test fixtures that are the descendants of the test may be run in parallel with
one another.`

Expand Down Expand Up @@ -67,8 +74,11 @@ to other tests at the same level in the tree. The following are a few examples o
3. __Non-parallel SetUpFixture with parallel test fixtures:__ The entire group of fixtures runs separately from any
fixtures outside the group. Within the group, multiple fixtures run in parallel.

4. __Parallel SetUpFixture with non-parallel test fixtures:__ The group runs in parallel with other fixtures and groups.
Within the group, only one fixture at a time may execute.
4. __Parallel SetUpFixture with non-parallel test fixtures:__ The SetUpFixture's own one-time setup and teardown may run
in parallel with other work, but the non-parallel fixtures inside it are queued together with every other
non-parallel fixture in the assembly and run one at a time, after all parallel work has finished. This does not
produce a group that runs in parallel with other groups while its own fixtures run in sequence; see (3) for the
way to isolate a group.

5. __Parallel SetUpFixture with non-parallel test fixtures containing parallel test cases:__ This is just one example of
a more complex setup. The fixtures themselves run as described in (4) but the cases within each fixture run in
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,12 @@ using NUnit.Framework;
3. Setting the level to 1 effectively disables parallelism (only one test runs at a time).
4. This value can be overridden using the `--workers` command-line option in the NUnit console runner.
5. Consider your test dependencies and shared resources when setting this value. Higher parallelism can cause issues with tests that share state.
6. The attribute cannot be applied to a single fixture or test. To limit how many cases of one test run at the same time, throttle inside the test itself, for example with a `SemaphoreSlim`. See [Running Tests in Parallel](xref:paralleltests#limiting-parallelism-for-one-test).

## See Also

* [Running Tests in Parallel](xref:paralleltests)
* [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md)
* [Parallelizable Attribute](xref:attribute-parallelizable)
* [NonParallelizable Attribute](xref:attribute-nonparallelizable)
* [Console Command Line](xref:consolecommandline)
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,7 @@ does not support parallelization.

## See Also

* [Running Tests in Parallel](xref:paralleltests)
* [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md)
* [Parallelizable Attribute](xref:attribute-parallelizable)
* [LevelOfParallelism Attribute](xref:attribute-levelofparallelism)
16 changes: 13 additions & 3 deletions docs/articles/nunit/writing-tests/attributes/parallelizable.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ The `ParallelizableAttribute` marks an assembly, fixture, or method (and optiona

When used without an argument, scope defaults to `ParallelScope.Self`.

> [!TIP]
> This page describes the attribute. For the common questions, such as how to turn parallel execution on, what to
> do about state shared between tests, and how to run fixtures in parallel while keeping each fixture's tests in
> order, see [Running Tests in Parallel](xref:paralleltests). How NUnit schedules parallel work is described in
> [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md).

## Constructors

```csharp
Expand Down Expand Up @@ -53,14 +59,16 @@ Named argument form:
| Value | Meaning | Valid On |
| ------------------------ | --------------------------------------------------------------------------------- | ----------------- |
| `ParallelScope.Self` | the test itself may be run in parallel with other tests | Classes, Methods |
| `ParallelScope.Children` | child tests may be run in parallel with one another | Assembly, Classes |
| `ParallelScope.Children` | child tests may be run in parallel with one another | Assembly, Classes, parameterized test methods |
| `ParallelScope.Fixtures` | fixtures may be run in parallel with one another | Assembly, Classes |
| `ParallelScope.All` | the test and its descendants may be run in parallel with others at the same level | Classes, Methods |
| `ParallelScope.All` | the test and its descendants may be run in parallel with others at the same level | Classes, parameterized test methods |

## Notes

* Some values are invalid on certain elements, although they will compile. NUnit will report any tests so marked as
invalid and will produce an error message.
invalid and will produce an error message. On a test method that is not parameterized, only `ParallelScope.Self`
is valid. On a parameterized test method, such as one with `[TestCase]` or `[TestCaseSource]`, `Children` or `All`
lets the individual test cases run in parallel with one another.

* The `ParallelScope` enum has additional values, which are used internally but are not visible to users through
Intellisense.
Expand All @@ -70,6 +78,8 @@ Named argument form:

## See Also

* [Running Tests in Parallel](xref:paralleltests)
* [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md)
* [FixtureLifeCycle Attribute](xref:attribute-fixturelifecycle)
* [NonParallelizable Attribute](xref:attribute-nonparallelizable)
* [LevelOfParallelism Attribute](xref:attribute-levelofparallelism)
162 changes: 162 additions & 0 deletions docs/articles/nunit/writing-tests/parallel-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
---
uid: paralleltests
---

# Running Tests in Parallel

Within a test assembly, NUnit runs tests one at a time unless you tell it otherwise. Marking a test
[`[Parallelizable]`](xref:attribute-parallelizable) is enough to have NUnit schedule it on a pool of worker threads,
alongside any other test marked the same way. The attribute is a statement that the test is *able* to run at the same
time as other tests. Making that true is your job, and most of this page is about that. It answers the questions that
come up most often. The attribute reference pages describe each attribute, and
[Framework Parallel Test Execution](../technical-notes/usage/Framework-Parallel-Test-Execution.md) describes how NUnit
schedules the work.

> [!TIP]
> Parallel execution only helps when the tests are independent. Most problems reported with parallel tests turn out to
> be state shared between tests, not a problem with NUnit. See
> [Tests that only fail when run in parallel](#tests-that-only-fail-when-run-in-parallel) before you reach for
> `[NonParallelizable]`.

## Turning parallel execution on

There are two ways to opt in, and choosing between them is the first decision to make.

**Mark fixtures and tests one at a time.** `[Parallelizable]` on a class means the fixture runs alongside other
fixtures while its own tests run in order. `[Parallelizable(ParallelScope.All)]` lets its tests run in parallel with
one another as well. Everything you have not marked keeps running one at a time, so each fixture you mark is one you
have checked.

[!code-csharp[ParallelFixtureSelf](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelFixtureSelf)]

**Turn it on for the whole assembly** and mark the exceptions `[NonParallelizable]`. The attribute usually goes in a
file such as `AssemblyInfo.cs` or `GlobalUsings.cs`:

```csharp
using NUnit.Framework;

[assembly: Parallelizable(ParallelScope.Fixtures)]
```

With `ParallelScope.Fixtures`, test fixtures run in parallel with one another, but the tests inside each fixture still
run one at a time. With `ParallelScope.Children`, every fixture and every test runs in parallel with everything else,
unless a fixture or test overrides that with its own attribute. This is the quickest way to get a parallel run, but
not necessarily the safest. Every fixture in the assembly becomes a candidate, including ones written before the
setting existed and ones added later, so a test that shares state shows up later as an intermittent failure rather
than at the moment you turned parallelism on.

Many teams start with the first approach and switch to the second once their fixtures are known to be independent.

NUnit uses as many worker threads as the machine has processors, with a minimum of two. The
[`[LevelOfParallelism]`](xref:attribute-levelofparallelism) assembly attribute changes that number.

## Tests that only fail when run in parallel

A test that passes on its own and fails when run in parallel is almost always sharing something mutable with another
test. The usual suspects are:

- **Fixture instance fields.** By default NUnit creates one instance of your fixture class and runs every test in it
on that instance. A `[SetUp]` method that assigns a field is overwriting the same field that another test, running on
another thread, is still using.
- **Static fields and singletons.** A static `HttpClient`, database connection, or browser driver is shared by every
test in the process, whatever fixture they are in.
- **External resources.** Two tests writing to the same file, the same database table, or the same port.

For fixture instance state, [`[FixtureLifeCycle(LifeCycle.InstancePerTestCase)]`](xref:attribute-fixturelifecycle)
gives each test its own instance of the fixture, so `[SetUp]` and `[TearDown]` work on a resource that no other test
can see. This example creates a session per test, which is also the pattern for a Selenium web driver or any other
object that must not be driven from two threads at once:

[!code-csharp[ParallelInstancePerTestCase](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelInstancePerTestCase)]

For static state and external resources, NUnit cannot isolate the tests for you. Either give each test its own copy of
the resource, or mark the tests that need it [`[NonParallelizable]`](xref:attribute-nonparallelizable) so that they run
on their own after all parallel work has finished:

[!code-csharp[ParallelNonParallelizable](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelNonParallelizable)]

> [!NOTE]
> `[OneTimeSetUp]` and `[OneTimeTearDown]` run once per fixture even with `InstancePerTestCase`, and must then be
> static. Anything they create is shared by every test in the fixture.

## Running fixtures in parallel but each fixture's tests in order

A fixture is the unit NUnit uses to keep tests in order. `[Parallelizable]` without an argument, which is
`ParallelScope.Self`, means the fixture may run at the same time as other fixtures while its own tests run one after
another. The same applies when the assembly uses `ParallelScope.Fixtures`. If a set of tests shares a resource that
must not be used from two threads at once, keeping those tests in one fixture marked `[Parallelizable]` is the simplest
way to serialize them without slowing down the rest of the run.

## Running the cases of one test in parallel

A test method with `[TestCase]`, `[TestCaseSource]` or `[Values]` arguments is a small suite of test cases. Mark it with
`[Parallelizable(ParallelScope.Children)]` and the cases run in parallel with one another. `ParallelScope.All` does
the same thing on a test method, because the suite itself always runs on its fixture's thread and only the `Children`
part has an effect. Whether the cases also overlap with other tests depends on the fixture. In a fixture that is not
itself parallel, the cases run in parallel with one another and nothing else runs at the same time, not even the
fixture's other tests. In a parallel fixture, the cases share the worker pool with everything else that is eligible.

[!code-csharp[ParallelTestCases](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelTestCases)]

On a test method that is not parameterized, that is a plain `[Test]` method with no `[TestCase]`, `[TestCaseSource]` or
`[Values]`, `ParallelScope.Self` is the only valid scope. NUnit reports the test as invalid if you use `Children` or
`All` there, and the NUnit Analyzers flag it at build time as
[NUnit1009](../../nunit-analyzers/NUnit1009.md).

## Keeping a group of fixtures apart from the rest

There is no attribute for a namespace, but a [`[SetUpFixture]`](xref:attribute-setupfixture) applies to every fixture
in its namespace, and its parallel settings apply to them as well. A `[NonParallelizable]` SetUpFixture with
`[Parallelizable]` fixtures inside it runs as a group. Nothing outside the group runs while the group runs, and within
the group the fixtures run in parallel with one another:

[!code-csharp[ParallelGroupIsolated](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelGroupIsolated)]

The opposite arrangement, a group that runs in parallel with other groups while its own fixtures run one at a time, is
not something NUnit supports. A `[Parallelizable]` SetUpFixture with `[NonParallelizable]` fixtures inside it does not
do that: every non-parallel fixture in the assembly waits in the same queue and runs one at a time, whatever namespace
it is in. If you need that shape, put the tests that share the resource in one fixture, as described above, or move
each group to its own test project and let the test runner run the projects in parallel. Whether a runner does that by
default is covered in [Parallel execution did not make the run faster](#parallel-execution-did-not-make-the-run-faster).

## Limiting parallelism for one test

[`[LevelOfParallelism]`](xref:attribute-levelofparallelism) sets the number of worker threads for the whole assembly and
cannot be applied to a single fixture or test. To let a parameterized test run its cases in parallel while limiting how
many run at once, for example because each case calls a rate-limited service, throttle inside the test with a static
`SemaphoreSlim`:

[!code-csharp[ParallelThrottle](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelThrottle)]

## Parallel execution did not make the run faster

Parallel execution has a cost. For a suite of very fast tests, starting and switching between worker threads can take
longer than the tests themselves, so adding `[Parallelizable]` may make such a run slightly slower rather than faster.
It pays off when the tests spend their time waiting, for a database, a web request, or a browser.

If your solution has several test projects, note that running them in parallel with one another is the job of the test
runner, not of NUnit, and each runner has its own default:

- The NUnit console runner runs each assembly in its own process, in parallel, by default. See
[Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md).
- `dotnet test` in its default VSTest mode, and Visual Studio Test Explorer, run one project at a time unless
`MaxCpuCount` is set in a runsettings file.
- `dotnet test` in Microsoft.Testing.Platform mode, which the .NET 10 SDK uses when `global.json` selects that runner,
runs test projects in parallel by default, up to the number of processors. `--max-parallel-test-modules` changes the
limit. See [NUnit and Microsoft.Testing.Platform](../../vs-test-adapter/NUnit-And-Microsoft-Test-Platform.md).

When a project with more than a couple of thousand tests runs much more slowly as a whole than the same tests run in
smaller batches, check the adapter's [`AssemblySelectLimit`](xref:tipsandtricks#assemblyselectlimit) setting before
looking at parallelism.

## See also

- The [Parallelizable](xref:attribute-parallelizable), [NonParallelizable](xref:attribute-nonparallelizable),
[LevelOfParallelism](xref:attribute-levelofparallelism) and [FixtureLifeCycle](xref:attribute-fixturelifecycle)
reference pages
- [Framework Parallel Test Execution](../technical-notes/usage/Framework-Parallel-Test-Execution.md), for how NUnit
schedules parallel and non-parallel work
- [Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md), for running assemblies
in parallel with the console runner
- [Preparing and Cleaning Up Tests](xref:setupandteardownguide), for which setup methods run per test and which run
once
2 changes: 2 additions & 0 deletions docs/articles/nunit/writing-tests/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
href: dependent-tests.md
- name: Flaky and Slow Tests
href: flaky-and-slow-tests.md
- name: Running Tests in Parallel
href: parallel-tests.md
- name: Organizing and Selecting Tests
href: organizing-tests.md
- name: Attributes
Expand Down
3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ _description: Documentation for NUnit, the open-source unit-testing framework fo
<a href="articles/nunit/writing-tests/setup-and-teardown.md">Preparing and cleaning up</a>
<a href="articles/nunit/writing-tests/dependent-tests.md">Tests that depend on each other</a>
<a href="articles/nunit/writing-tests/flaky-and-slow-tests.md">Flaky tests</a>
<a href="articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md">Running tests in parallel</a>
<a href="articles/nunit/writing-tests/parallel-tests.md">Running tests in parallel</a>
<a href="articles/vs-test-adapter/NUnit-And-Microsoft-Test-Platform.md">Microsoft.Testing.Platform</a>
<a href="articles/nunit/V5BreakingChanges.md">Upgrading to NUnit 5</a>
<a href="articles/nunit-analyzers/NUnit-Analyzers.md">Analyzer warnings</a>
Expand Down Expand Up @@ -64,6 +64,7 @@ public class CalculatorTests
<li><a href="articles/nunit/writing-tests/assertions/multiple-asserts.md">Checking several things at once</a></li>
<li><a href="articles/nunit/writing-tests/dependent-tests.md">Tests that depend on other tests</a></li>
<li><a href="articles/nunit/writing-tests/flaky-and-slow-tests.md">Flaky and slow tests</a></li>
<li><a href="articles/nunit/writing-tests/parallel-tests.md">Running tests in parallel</a></li>
<li><a href="articles/nunit/writing-tests/organizing-tests.md">Organizing and selecting tests</a></li>
</ul>
<h3>Reference</h3>
Expand Down
Loading
Loading