From 6e1cd77919c2f47fde052f7e42a580cc1ed5edd4 Mon Sep 17 00:00:00 2001 From: Sean Killeen Date: Fri, 23 Oct 2026 16:11:42 -0400 Subject: [PATCH 1/4] Add a Running Tests in Parallel guide and link it from the reference pages Closes #272. Adds docs/articles/nunit/writing-tests/parallel-tests.md, a task-oriented topic page in the same style as Flaky and Slow Tests, covering the questions that come up most often in nunit/nunit discussions: turning parallel execution on, tests that only fail in parallel (shared state), fixtures in parallel with tests in order, parallel test cases, isolating a group of fixtures, throttling one test, and why a parallel run can be slower. All examples are executable snippets in ParallelTestsGuideExamples.cs, plus a verification fixture that builds test methods the way NUnit does to prove which scopes are valid. The Parallelizable, NonParallelizable and LevelOfParallelism reference pages now point to the guide and to the Framework Parallel Test Execution note, and the home page links the guide instead of the technical note. Corrections found while verifying the claims against NUnit 5.0.0: - ParallelScope.All and Children are invalid on a non-parameterized test method but valid on a parameterized one; the reference table said All was valid on methods. - Framework Parallel Test Execution case 4 claimed a Parallelizable SetUpFixture with NonParallelizable fixtures runs as a group in parallel with other groups. A timing experiment shows every non-parallel fixture queues on the single non-parallel worker regardless of namespace, so the case now describes what actually happens and points to case 3. Co-Authored-By: Claude Fable 5.1 --- .../Framework-Parallel-Test-Execution.md | 14 +- .../attributes/levelofparallelism.md | 3 + .../attributes/nonparallelizable.md | 2 + .../attributes/parallelizable.md | 16 +- .../nunit/writing-tests/parallel-tests.md | 147 +++++++++++ docs/articles/nunit/writing-tests/toc.yml | 2 + docs/index.md | 3 +- .../ParallelTestsGuideExamples.cs | 246 ++++++++++++++++++ 8 files changed, 426 insertions(+), 7 deletions(-) create mode 100644 docs/articles/nunit/writing-tests/parallel-tests.md create mode 100644 docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs diff --git a/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md b/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md index 073573dd3..d4d70588a 100644 --- a/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md +++ b/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md @@ -7,6 +7,11 @@ test run. By default, no parallel execution takes place. Attributes are used to indicate which tests may run in parallel and how they relate to other tests. +> [!TIP] +> This note describes how NUnit schedules parallel work. For practical questions, such as how to turn parallel execution +> on and what to do about state shared between tests, see +> [Running Tests in Parallel](../../writing-tests/parallel-tests.md). + ## Platform Support Parallel execution is supported by the NUnit framework on desktop .NET runtimes and .NET Standard 2.0. It is not @@ -36,7 +41,7 @@ This is a `[Flags]` type enumeration used to specify which tests may run in para it appears and any subordinate tests. The following values are available for use: * `ParallelScope.Self` indicates that the test itself may be run in parallel with other tests. This is the default for - the `ParallelizableAttribute` and is the only value permitted on a test method. + the `ParallelizableAttribute` and is the only value permitted on a test method that is not parameterized. * `ParallelScope.Children` indicates that the descendants of the test may be run in parallel with respect to one another. * `ParallelScope.Fixtures` indicates that test fixtures that are the descendants of the test may be run in parallel with @@ -67,8 +72,11 @@ to other tests at the same level in the tree. The following are a few examples o 3. __Non-parallel SetUpFixture with parallel test fixtures:__ The entire group of fixtures runs separately from any fixtures outside the group. Within the group, multiple fixtures run in parallel. -4. __Parallel SetUpFixture with non-parallel test fixtures:__ The group runs in parallel with other fixtures and groups. - Within the group, only one fixture at a time may execute. +4. __Parallel SetUpFixture with non-parallel test fixtures:__ The SetUpFixture's own one-time setup and teardown may run + in parallel with other work, but the non-parallel fixtures inside it are queued together with every other + non-parallel fixture in the assembly and run one at a time, after all parallel work has finished. This does not + produce a group that runs in parallel with other groups while its own fixtures run in sequence; see (3) for the + way to isolate a group. 5. __Parallel SetUpFixture with non-parallel test fixtures containing parallel test cases:__ This is just one example of a more complex setup. The fixtures themselves run as described in (4) but the cases within each fixture run in diff --git a/docs/articles/nunit/writing-tests/attributes/levelofparallelism.md b/docs/articles/nunit/writing-tests/attributes/levelofparallelism.md index 66eed190d..80bbf361d 100644 --- a/docs/articles/nunit/writing-tests/attributes/levelofparallelism.md +++ b/docs/articles/nunit/writing-tests/attributes/levelofparallelism.md @@ -45,9 +45,12 @@ using NUnit.Framework; 3. Setting the level to 1 effectively disables parallelism (only one test runs at a time). 4. This value can be overridden using the `--workers` command-line option in the NUnit console runner. 5. Consider your test dependencies and shared resources when setting this value. Higher parallelism can cause issues with tests that share state. +6. The attribute cannot be applied to a single fixture or test. To limit how many cases of one test run at the same time, throttle inside the test itself, for example with a `SemaphoreSlim`. See [Running Tests in Parallel](xref:paralleltests#limiting-parallelism-for-one-test). ## See Also +* [Running Tests in Parallel](xref:paralleltests) +* [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md) * [Parallelizable Attribute](xref:attribute-parallelizable) * [NonParallelizable Attribute](xref:attribute-nonparallelizable) * [Console Command Line](xref:consolecommandline) diff --git a/docs/articles/nunit/writing-tests/attributes/nonparallelizable.md b/docs/articles/nunit/writing-tests/attributes/nonparallelizable.md index 7ffd5ab23..6d1b53692 100644 --- a/docs/articles/nunit/writing-tests/attributes/nonparallelizable.md +++ b/docs/articles/nunit/writing-tests/attributes/nonparallelizable.md @@ -30,5 +30,7 @@ does not support parallelization. ## See Also +* [Running Tests in Parallel](xref:paralleltests) +* [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md) * [Parallelizable Attribute](xref:attribute-parallelizable) * [LevelOfParallelism Attribute](xref:attribute-levelofparallelism) diff --git a/docs/articles/nunit/writing-tests/attributes/parallelizable.md b/docs/articles/nunit/writing-tests/attributes/parallelizable.md index 64556d9bf..0284f308d 100644 --- a/docs/articles/nunit/writing-tests/attributes/parallelizable.md +++ b/docs/articles/nunit/writing-tests/attributes/parallelizable.md @@ -10,6 +10,12 @@ The `ParallelizableAttribute` marks an assembly, fixture, or method (and optiona When used without an argument, scope defaults to `ParallelScope.Self`. +> [!TIP] +> This page describes the attribute. For the common questions, such as how to turn parallel execution on, what to +> do about state shared between tests, and how to run fixtures in parallel while keeping each fixture's tests in +> order, see [Running Tests in Parallel](xref:paralleltests). How NUnit schedules parallel work is described in +> [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md). + ## Constructors ```csharp @@ -53,14 +59,16 @@ Named argument form: | Value | Meaning | Valid On | | ------------------------ | --------------------------------------------------------------------------------- | ----------------- | | `ParallelScope.Self` | the test itself may be run in parallel with other tests | Classes, Methods | -| `ParallelScope.Children` | child tests may be run in parallel with one another | Assembly, Classes | +| `ParallelScope.Children` | child tests may be run in parallel with one another | Assembly, Classes, parameterized test methods | | `ParallelScope.Fixtures` | fixtures may be run in parallel with one another | Assembly, Classes | -| `ParallelScope.All` | the test and its descendants may be run in parallel with others at the same level | Classes, Methods | +| `ParallelScope.All` | the test and its descendants may be run in parallel with others at the same level | Classes, parameterized test methods | ## Notes * Some values are invalid on certain elements, although they will compile. NUnit will report any tests so marked as - invalid and will produce an error message. + invalid and will produce an error message. On a test method that is not parameterized, only `ParallelScope.Self` + is valid. On a parameterized test method, such as one with `[TestCase]` or `[TestCaseSource]`, `Children` or `All` + lets the individual test cases run in parallel with one another. * The `ParallelScope` enum has additional values, which are used internally but are not visible to users through Intellisense. @@ -70,6 +78,8 @@ Named argument form: ## See Also +* [Running Tests in Parallel](xref:paralleltests) +* [Framework Parallel Test Execution](../../technical-notes/usage/Framework-Parallel-Test-Execution.md) * [FixtureLifeCycle Attribute](xref:attribute-fixturelifecycle) * [NonParallelizable Attribute](xref:attribute-nonparallelizable) * [LevelOfParallelism Attribute](xref:attribute-levelofparallelism) diff --git a/docs/articles/nunit/writing-tests/parallel-tests.md b/docs/articles/nunit/writing-tests/parallel-tests.md new file mode 100644 index 000000000..343b1e817 --- /dev/null +++ b/docs/articles/nunit/writing-tests/parallel-tests.md @@ -0,0 +1,147 @@ +--- +uid: paralleltests +--- + +# Running Tests in Parallel + +NUnit runs tests one at a time unless you tell it otherwise. The [`[Parallelizable]`](xref:attribute-parallelizable) +attribute does not tell NUnit to do anything by itself. It is a promise from you that the test is able to run at the +same time as other tests, and NUnit then schedules it on one of a pool of worker threads. This page answers the +questions that come up most often. The attribute reference pages describe each attribute, and +[Framework Parallel Test Execution](../technical-notes/usage/Framework-Parallel-Test-Execution.md) describes how NUnit +schedules the work. + +> [!TIP] +> Parallel execution only helps when the tests are independent. Most problems reported with parallel tests turn out to +> be state shared between tests, not a problem with NUnit. See +> [Tests that only fail when run in parallel](#tests-that-only-fail-when-run-in-parallel) before you reach for +> `[NonParallelizable]`. + +## Turning parallel execution on + +The quickest way to opt in is an assembly-level attribute, usually placed in a file such as `AssemblyInfo.cs` or +`GlobalUsings.cs`: + +```csharp +using NUnit.Framework; + +[assembly: Parallelizable(ParallelScope.Fixtures)] +``` + +With `ParallelScope.Fixtures`, test fixtures run in parallel with one another, but the tests inside each fixture still +run one at a time. With `ParallelScope.Children`, every fixture and every test runs in parallel with everything else, +unless a fixture or test overrides that with its own attribute. NUnit itself makes no recommendation between the two: +`Fixtures` is a smaller step if your fixtures hold state, `Children` gives the most parallelism if your tests are fully +independent. + +You can instead opt in one fixture at a time. `[Parallelizable]` on a class means the fixture runs alongside other +fixtures while its own tests run in order. Use `[Parallelizable(ParallelScope.All)]` on the class to let its tests run +in parallel with one another as well. + +[!code-csharp[ParallelFixtureSelf](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelFixtureSelf)] + +NUnit uses as many worker threads as the machine has processors, with a minimum of two. The +[`[LevelOfParallelism]`](xref:attribute-levelofparallelism) assembly attribute changes that number. + +## Tests that only fail when run in parallel + +A test that passes on its own and fails when run in parallel is almost always sharing something mutable with another +test. The usual suspects are: + +- **Fixture instance fields.** By default NUnit creates one instance of your fixture class and runs every test in it + on that instance. A `[SetUp]` method that assigns a field is overwriting the same field that another test, running on + another thread, is still using. +- **Static fields and singletons.** A static `HttpClient`, database connection, or browser driver is shared by every + test in the process, whatever fixture they are in. +- **External resources.** Two tests writing to the same file, the same database table, or the same port. + +For fixture instance state, [`[FixtureLifeCycle(LifeCycle.InstancePerTestCase)]`](xref:attribute-fixturelifecycle) +gives each test its own instance of the fixture, so `[SetUp]` and `[TearDown]` work on a resource that no other test +can see. This example creates a session per test, which is also the pattern for a Selenium web driver or any other +object that must not be driven from two threads at once: + +[!code-csharp[ParallelInstancePerTestCase](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelInstancePerTestCase)] + +For static state and external resources, NUnit cannot isolate the tests for you. Either give each test its own copy of +the resource, or mark the tests that need it [`[NonParallelizable]`](xref:attribute-nonparallelizable) so that they run +on their own after all parallel work has finished: + +[!code-csharp[ParallelNonParallelizable](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelNonParallelizable)] + +> [!NOTE] +> `[OneTimeSetUp]` and `[OneTimeTearDown]` run once per fixture even with `InstancePerTestCase`, and must then be +> static. Anything they create is shared by every test in the fixture. + +## Running fixtures in parallel but each fixture's tests in order + +A fixture is the unit NUnit uses to keep tests in order. `[Parallelizable]` without an argument, which is +`ParallelScope.Self`, means the fixture may run at the same time as other fixtures while its own tests run one after +another. The same applies when the assembly uses `ParallelScope.Fixtures`. If a set of tests shares a resource that +must not be used from two threads at once, keeping those tests in one fixture marked `[Parallelizable]` is the simplest +way to serialize them without slowing down the rest of the run. + +## Running the cases of one test in parallel + +A test method with `[TestCase]`, `[TestCaseSource]` or `[Values]` arguments is a small suite of test cases. Mark it with +`[Parallelizable(ParallelScope.All)]` and the cases run in parallel with one another, even when the fixture runs its +other tests in order: + +[!code-csharp[ParallelTestCases](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelTestCases)] + +On a test method that has no arguments, `ParallelScope.Self` is the only valid scope. NUnit reports the test as invalid +if you use `Children` or `All` there, and the NUnit Analyzers flag it at build time as +[NUnit1009](../../nunit-analyzers/NUnit1009.md). + +## Keeping a group of fixtures apart from the rest + +There is no attribute for a namespace, but a [`[SetUpFixture]`](xref:attribute-setupfixture) applies to every fixture +in its namespace, and its parallel settings apply to them as well. A `[NonParallelizable]` SetUpFixture with +`[Parallelizable]` fixtures inside it runs as a group. Nothing outside the group runs while the group runs, and within +the group the fixtures run in parallel with one another: + +[!code-csharp[ParallelGroupIsolated](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelGroupIsolated)] + +The opposite arrangement, a group that runs in parallel with other groups while its own fixtures run one at a time, is +not something NUnit supports. A `[Parallelizable]` SetUpFixture with `[NonParallelizable]` fixtures inside it does not +do that: every non-parallel fixture in the assembly waits in the same queue and runs one at a time, whatever namespace +it is in. If you need that shape, put the tests that share the resource in one fixture, as described above, or move +each group to its own test project and let the test runner run the projects in parallel. `dotnet test` and Visual +Studio only do that when `MaxCpuCount` is set in a runsettings file, see +[Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md) for the NUnit console +runner. + +## Limiting parallelism for one test + +[`[LevelOfParallelism]`](xref:attribute-levelofparallelism) sets the number of worker threads for the whole assembly and +cannot be applied to a single fixture or test. To let a parameterized test run its cases in parallel while limiting how +many run at once, for example because each case calls a rate-limited service, throttle inside the test with a static +`SemaphoreSlim`: + +[!code-csharp[ParallelThrottle](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelThrottle)] + +## Parallel execution did not make the run faster + +Parallel execution has a cost. For a suite of very fast tests, starting and switching between worker threads can take +longer than the tests themselves, so adding `[Parallelizable]` may make such a run slightly slower rather than faster. +It pays off when the tests spend their time waiting, for a database, a web request, or a browser. + +If your solution has several test projects, note that running them in parallel with one another is the job of the test +runner, not of NUnit. The NUnit console runner does this by default, see +[Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md). `dotnet test` and +Visual Studio run one project at a time unless `MaxCpuCount` is set in a runsettings file. + +When a project with more than a couple of thousand tests runs much more slowly as a whole than the same tests run in +smaller batches, check the adapter's [`AssemblySelectLimit`](xref:tipsandtricks#assemblyselectlimit) setting before +looking at parallelism. + +## See also + +- The [Parallelizable](xref:attribute-parallelizable), [NonParallelizable](xref:attribute-nonparallelizable), + [LevelOfParallelism](xref:attribute-levelofparallelism) and [FixtureLifeCycle](xref:attribute-fixturelifecycle) + reference pages +- [Framework Parallel Test Execution](../technical-notes/usage/Framework-Parallel-Test-Execution.md), for how NUnit + schedules parallel and non-parallel work +- [Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md), for running assemblies + in parallel with the console runner +- [Preparing and Cleaning Up Tests](xref:setupandteardownguide), for which setup methods run per test and which run + once diff --git a/docs/articles/nunit/writing-tests/toc.yml b/docs/articles/nunit/writing-tests/toc.yml index ecb896df6..852a685ef 100644 --- a/docs/articles/nunit/writing-tests/toc.yml +++ b/docs/articles/nunit/writing-tests/toc.yml @@ -10,6 +10,8 @@ href: dependent-tests.md - name: Flaky and Slow Tests href: flaky-and-slow-tests.md +- name: Running Tests in Parallel + href: parallel-tests.md - name: Organizing and Selecting Tests href: organizing-tests.md - name: Attributes diff --git a/docs/index.md b/docs/index.md index 7cede9921..c485717e9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -29,7 +29,7 @@ _description: Documentation for NUnit, the open-source unit-testing framework fo Preparing and cleaning up Tests that depend on each other Flaky tests -Running tests in parallel +Running tests in parallel Microsoft.Testing.Platform Upgrading to NUnit 5 Analyzer warnings @@ -64,6 +64,7 @@ public class CalculatorTests
  • Checking several things at once
  • Tests that depend on other tests
  • Flaky and slow tests
    • +
  • Running tests in parallel
  • Organizing and selecting tests

    • Reference

    diff --git a/docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs b/docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs new file mode 100644 index 000000000..af254b47a --- /dev/null +++ b/docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs @@ -0,0 +1,246 @@ +using System.Collections.Concurrent; +using NUnit.Framework; +using NUnit.Framework.Interfaces; +using NUnit.Framework.Internal.Builders; +using MethodWrapper = NUnit.Framework.Internal.MethodWrapper; +using ParameterizedMethodSuite = NUnit.Framework.Internal.ParameterizedMethodSuite; +using PropertyNames = NUnit.Framework.Internal.PropertyNames; + +#pragma warning disable CA1822 + +namespace Snippets.NUnit.ParallelGuide +{ + /// + /// Stands in for a resource that must not be used from two threads at once, + /// such as a Selenium web driver or a database connection. + /// + public sealed class Session : IDisposable + { + public int CallCount { get; private set; } + + public string Call(string input) + { + CallCount++; + return input.ToUpperInvariant(); + } + + public void Dispose() + { + } + } + + #region ParallelFixtureSelf + // Runs alongside other fixtures; its two tests run one after the other. + [Parallelizable] + public class OrderCalculatorTests + { + [Test] + public void Total_AddsLineItems() + { + Assert.That(2 + 3, Is.EqualTo(5)); + } + + [Test] + public void Total_IsZeroForEmptyOrder() + { + Assert.That(0, Is.EqualTo(0)); + } + } + #endregion + + #region ParallelInstancePerTestCase + [Parallelizable(ParallelScope.All)] + [FixtureLifeCycle(LifeCycle.InstancePerTestCase)] + public class SessionTests + { + private Session _session = null!; + + [SetUp] + public void OpenSession() + { + // Each test gets its own fixture instance, so each test gets its own session. + _session = new Session(); + } + + [TearDown] + public void CloseSession() + { + _session.Dispose(); + } + + [Test] + public void Call_ReturnsUpperCase() + { + Assert.That(_session.Call("abc"), Is.EqualTo("ABC")); + } + + [Test] + public void Call_CountsCalls() + { + _session.Call("x"); + _session.Call("y"); + + // Would be unreliable if another test on another thread shared this session. + Assert.That(_session.CallCount, Is.EqualTo(2)); + } + } + #endregion + + #region ParallelNonParallelizable + public static class SharedCache + { + public static readonly ConcurrentDictionary Entries = new(); + } + + // Uses a static resource, so it must not overlap with anything else that touches it. + [NonParallelizable] + public class SharedCacheTests + { + [SetUp] + public void ClearCache() + { + SharedCache.Entries.Clear(); + } + + [Test] + public void Cache_StartsEmpty() + { + Assert.That(SharedCache.Entries, Is.Empty); + } + + [Test] + public void Cache_StoresEntry() + { + SharedCache.Entries["key"] = "value"; + + Assert.That(SharedCache.Entries, Has.Count.EqualTo(1)); + } + } + #endregion + + #region ParallelTestCases + public class SlugTests + { + // The three cases of this test run in parallel with one another. + [TestCase("Hello World", "hello-world")] + [TestCase(" NUnit ", "nunit")] + [TestCase("A/B", "a-b")] + [Parallelizable(ParallelScope.All)] + public void ToSlug_NormalizesInput(string input, string expected) + { + var slug = input.Trim().ToLowerInvariant().Replace(' ', '-').Replace('/', '-'); + + Assert.That(slug, Is.EqualTo(expected)); + } + } + #endregion + + #region ParallelThrottle + public class RateLimitedServiceTests + { + // At most two cases call the service at the same time, whatever the worker count. + private static readonly SemaphoreSlim Throttle = new(2); + + [TestCase(1)] + [TestCase(2)] + [TestCase(3)] + [TestCase(4)] + [Parallelizable(ParallelScope.All)] + public async Task Service_HandlesRequest(int request) + { + await Throttle.WaitAsync(); + try + { + await Task.Delay(10); + Assert.That(request, Is.Positive); + } + finally + { + Throttle.Release(); + } + } + } + #endregion +} + +#region ParallelGroupIsolated +namespace Snippets.NUnit.ParallelGuide.Reporting +{ + // Nothing outside this namespace runs while the fixtures in it run. + [SetUpFixture] + [NonParallelizable] + public class ReportingSetUp + { + [OneTimeSetUp] + public void StartReportServer() + { + } + + [OneTimeTearDown] + public void StopReportServer() + { + } + } + + // Within the group, these two fixtures run in parallel with each other. + [Parallelizable] + public class DailyReportTests + { + [Test] + public void Report_HasHeader() => Assert.Pass(); + } + + [Parallelizable] + public class MonthlyReportTests + { + [Test] + public void Report_HasTotals() => Assert.Pass(); + } +} +#endregion + +namespace Snippets.NUnit.ParallelGuide +{ + /// + /// Not referenced from the docs. Checks the claims the guide makes about which scopes + /// are valid on a test method, by building the tests the way NUnit does. + /// + public class ParallelScopeVerification + { + private static class Holder + { + // Not a test: no [Test] attribute, so NUnit never discovers it. The analyzer + // still flags the invalid scope, which is the point being verified here. +#pragma warning disable NUnit1009 + [Parallelizable(ParallelScope.All)] + public static void PlainMethodWithAll() + { + } +#pragma warning restore NUnit1009 + } + + [Test] + public void AllOnPlainTestMethod_IsNotRunnable() + { + var method = new MethodWrapper(typeof(Holder), nameof(Holder.PlainMethodWithAll)); + + var test = new NUnitTestCaseBuilder().BuildTestMethod(method, null, null); + + Assert.That(test.RunState, Is.EqualTo(RunState.NotRunnable)); + Assert.That(test.Properties.Get(PropertyNames.SkipReason), + Is.EqualTo("May not specify ParallelScope.Children on a non-parameterized test method")); + } + + [Test] + public void AllOnParameterizedTestMethod_IsRunnable() + { + var method = new MethodWrapper(typeof(SlugTests), nameof(SlugTests.ToSlug_NormalizesInput)); + + var suite = new DefaultTestCaseBuilder().BuildFrom(method, null); + + Assert.That(suite, Is.TypeOf()); + Assert.That(suite.RunState, Is.EqualTo(RunState.Runnable)); + Assert.That(suite.Tests, Has.Count.EqualTo(3)); + } + } +} From bde3686e23fccfe5fe20ca3a9402feda7d1a1750 Mon Sep 17 00:00:00 2001 From: Sean Killeen Date: Fri, 23 Oct 2026 16:26:20 -0400 Subject: [PATCH 2/4] Scope the opening sentence of the parallel guide to a single assembly The NUnit console runner runs separate assemblies in parallel by default, so the unqualified claim only holds for the framework within one assembly. Co-Authored-By: Claude Fable 5.1 --- docs/articles/nunit/writing-tests/parallel-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/articles/nunit/writing-tests/parallel-tests.md b/docs/articles/nunit/writing-tests/parallel-tests.md index 343b1e817..bb784216f 100644 --- a/docs/articles/nunit/writing-tests/parallel-tests.md +++ b/docs/articles/nunit/writing-tests/parallel-tests.md @@ -4,7 +4,7 @@ uid: paralleltests # Running Tests in Parallel -NUnit runs tests one at a time unless you tell it otherwise. The [`[Parallelizable]`](xref:attribute-parallelizable) +Within a test assembly, NUnit runs tests one at a time unless you tell it otherwise. The [`[Parallelizable]`](xref:attribute-parallelizable) attribute does not tell NUnit to do anything by itself. It is a promise from you that the test is able to run at the same time as other tests, and NUnit then schedules it on one of a pool of worker threads. This page answers the questions that come up most often. The attribute reference pages describe each attribute, and From 1ff4857bc76e017df52f7565a1509ecd78c03ed4 Mon Sep 17 00:00:00 2001 From: Sean Killeen Date: Fri, 23 Oct 2026 20:11:30 -0400 Subject: [PATCH 3/4] Address Copilot review on the parallel guide - Say that [Parallelizable] is enough to get parallel scheduling, and separate that from making the test thread-safe. - Describe the Self-only restriction in terms of non-parameterized test methods rather than argument count. - Distinguish runner defaults: console runner parallel by default, VSTest mode needs MaxCpuCount, Microsoft.Testing.Platform mode of dotnet test runs modules in parallel by default (--max-parallel-test-modules). Co-Authored-By: Claude Fable 5.1 --- .../nunit/writing-tests/parallel-tests.md | 32 +++++++++++-------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/docs/articles/nunit/writing-tests/parallel-tests.md b/docs/articles/nunit/writing-tests/parallel-tests.md index bb784216f..568ec34e8 100644 --- a/docs/articles/nunit/writing-tests/parallel-tests.md +++ b/docs/articles/nunit/writing-tests/parallel-tests.md @@ -4,10 +4,11 @@ uid: paralleltests # Running Tests in Parallel -Within a test assembly, NUnit runs tests one at a time unless you tell it otherwise. The [`[Parallelizable]`](xref:attribute-parallelizable) -attribute does not tell NUnit to do anything by itself. It is a promise from you that the test is able to run at the -same time as other tests, and NUnit then schedules it on one of a pool of worker threads. This page answers the -questions that come up most often. The attribute reference pages describe each attribute, and +Within a test assembly, NUnit runs tests one at a time unless you tell it otherwise. Marking a test +[`[Parallelizable]`](xref:attribute-parallelizable) is enough to have NUnit schedule it on a pool of worker threads, +alongside any other test marked the same way. The attribute is a statement that the test is *able* to run at the same +time as other tests. Making that true is your job, and most of this page is about that. It answers the questions that +come up most often. The attribute reference pages describe each attribute, and [Framework Parallel Test Execution](../technical-notes/usage/Framework-Parallel-Test-Execution.md) describes how NUnit schedules the work. @@ -88,8 +89,9 @@ other tests in order: [!code-csharp[ParallelTestCases](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelTestCases)] -On a test method that has no arguments, `ParallelScope.Self` is the only valid scope. NUnit reports the test as invalid -if you use `Children` or `All` there, and the NUnit Analyzers flag it at build time as +On a test method that is not parameterized, that is a plain `[Test]` method with no `[TestCase]`, `[TestCaseSource]` or +`[Values]`, `ParallelScope.Self` is the only valid scope. NUnit reports the test as invalid if you use `Children` or +`All` there, and the NUnit Analyzers flag it at build time as [NUnit1009](../../nunit-analyzers/NUnit1009.md). ## Keeping a group of fixtures apart from the rest @@ -105,10 +107,8 @@ The opposite arrangement, a group that runs in parallel with other groups while not something NUnit supports. A `[Parallelizable]` SetUpFixture with `[NonParallelizable]` fixtures inside it does not do that: every non-parallel fixture in the assembly waits in the same queue and runs one at a time, whatever namespace it is in. If you need that shape, put the tests that share the resource in one fixture, as described above, or move -each group to its own test project and let the test runner run the projects in parallel. `dotnet test` and Visual -Studio only do that when `MaxCpuCount` is set in a runsettings file, see -[Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md) for the NUnit console -runner. +each group to its own test project and let the test runner run the projects in parallel. Whether a runner does that by +default is covered in [Parallel execution did not make the run faster](#parallel-execution-did-not-make-the-run-faster). ## Limiting parallelism for one test @@ -126,9 +126,15 @@ longer than the tests themselves, so adding `[Parallelizable]` may make such a r It pays off when the tests spend their time waiting, for a database, a web request, or a browser. If your solution has several test projects, note that running them in parallel with one another is the job of the test -runner, not of NUnit. The NUnit console runner does this by default, see -[Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md). `dotnet test` and -Visual Studio run one project at a time unless `MaxCpuCount` is set in a runsettings file. +runner, not of NUnit, and each runner has its own default: + +- The NUnit console runner runs each assembly in its own process, in parallel, by default. See + [Engine Parallel Test Execution](../technical-notes/usage/Engine-Parallel-Test-Execution.md). +- `dotnet test` in its default VSTest mode, and Visual Studio Test Explorer, run one project at a time unless + `MaxCpuCount` is set in a runsettings file. +- `dotnet test` in Microsoft.Testing.Platform mode, which the .NET 10 SDK uses when `global.json` selects that runner, + runs test projects in parallel by default, up to the number of processors. `--max-parallel-test-modules` changes the + limit. See [NUnit and Microsoft.Testing.Platform](../../vs-test-adapter/NUnit-And-Microsoft-Test-Platform.md). When a project with more than a couple of thousand tests runs much more slowly as a whole than the same tests run in smaller batches, check the adapter's [`AssemblySelectLimit`](xref:tipsandtricks#assemblyselectlimit) setting before From 40a3ee289481df88e74ab5660ccdff5710f46a26 Mon Sep 17 00:00:00 2001 From: Sean Killeen Date: Sat, 24 Oct 2026 11:03:45 -0400 Subject: [PATCH 4/4] Address review from manfred-brands and CharliePoole - Turning parallel execution on now presents the two approaches, per fixture first and assembly-wide second, states the trade-off of the assembly-wide setting, and notes that teams commonly start with the first. - Running the cases of one test in parallel uses ParallelScope.Children, explains that All is equivalent on a test method, and describes how the fixture decides whether the cases overlap with anything else. Verified by timing runs: in a non-parallel fixture the cases overlapped only each other; under an assembly-level Fixtures setting they overlapped sibling tests and other fixtures, identically for Children and All. - Technical note records why: a parameterized method's suite always runs directly on its fixture's thread, so only Children has an effect there. Co-Authored-By: Claude Fable 5.1 --- .../Framework-Parallel-Test-Execution.md | 4 ++- .../nunit/writing-tests/parallel-tests.md | 33 ++++++++++++------- .../ParallelTestsGuideExamples.cs | 6 ++-- 3 files changed, 27 insertions(+), 16 deletions(-) diff --git a/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md b/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md index d4d70588a..a99e1c8cf 100644 --- a/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md +++ b/docs/articles/nunit/technical-notes/usage/Framework-Parallel-Test-Execution.md @@ -43,7 +43,9 @@ it appears and any subordinate tests. The following values are available for use * `ParallelScope.Self` indicates that the test itself may be run in parallel with other tests. This is the default for the `ParallelizableAttribute` and is the only value permitted on a test method that is not parameterized. * `ParallelScope.Children` indicates that the descendants of the test may be run in parallel with respect to one - another. + another. On a parameterized test method this is the only part of the scope that has an effect: the suite of test + cases always executes directly on the thread of its fixture, so `Self` is ignored there and `Children` and `All` are + equivalent. The individual cases are then dispatched to the parallel queue. * `ParallelScope.Fixtures` indicates that test fixtures that are the descendants of the test may be run in parallel with one another.` diff --git a/docs/articles/nunit/writing-tests/parallel-tests.md b/docs/articles/nunit/writing-tests/parallel-tests.md index 568ec34e8..156b52249 100644 --- a/docs/articles/nunit/writing-tests/parallel-tests.md +++ b/docs/articles/nunit/writing-tests/parallel-tests.md @@ -20,8 +20,17 @@ schedules the work. ## Turning parallel execution on -The quickest way to opt in is an assembly-level attribute, usually placed in a file such as `AssemblyInfo.cs` or -`GlobalUsings.cs`: +There are two ways to opt in, and choosing between them is the first decision to make. + +**Mark fixtures and tests one at a time.** `[Parallelizable]` on a class means the fixture runs alongside other +fixtures while its own tests run in order. `[Parallelizable(ParallelScope.All)]` lets its tests run in parallel with +one another as well. Everything you have not marked keeps running one at a time, so each fixture you mark is one you +have checked. + +[!code-csharp[ParallelFixtureSelf](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelFixtureSelf)] + +**Turn it on for the whole assembly** and mark the exceptions `[NonParallelizable]`. The attribute usually goes in a +file such as `AssemblyInfo.cs` or `GlobalUsings.cs`: ```csharp using NUnit.Framework; @@ -31,15 +40,12 @@ using NUnit.Framework; With `ParallelScope.Fixtures`, test fixtures run in parallel with one another, but the tests inside each fixture still run one at a time. With `ParallelScope.Children`, every fixture and every test runs in parallel with everything else, -unless a fixture or test overrides that with its own attribute. NUnit itself makes no recommendation between the two: -`Fixtures` is a smaller step if your fixtures hold state, `Children` gives the most parallelism if your tests are fully -independent. - -You can instead opt in one fixture at a time. `[Parallelizable]` on a class means the fixture runs alongside other -fixtures while its own tests run in order. Use `[Parallelizable(ParallelScope.All)]` on the class to let its tests run -in parallel with one another as well. +unless a fixture or test overrides that with its own attribute. This is the quickest way to get a parallel run, but +not necessarily the safest. Every fixture in the assembly becomes a candidate, including ones written before the +setting existed and ones added later, so a test that shares state shows up later as an intermittent failure rather +than at the moment you turned parallelism on. -[!code-csharp[ParallelFixtureSelf](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelFixtureSelf)] +Many teams start with the first approach and switch to the second once their fixtures are known to be independent. NUnit uses as many worker threads as the machine has processors, with a minimum of two. The [`[LevelOfParallelism]`](xref:attribute-levelofparallelism) assembly attribute changes that number. @@ -84,8 +90,11 @@ way to serialize them without slowing down the rest of the run. ## Running the cases of one test in parallel A test method with `[TestCase]`, `[TestCaseSource]` or `[Values]` arguments is a small suite of test cases. Mark it with -`[Parallelizable(ParallelScope.All)]` and the cases run in parallel with one another, even when the fixture runs its -other tests in order: +`[Parallelizable(ParallelScope.Children)]` and the cases run in parallel with one another. `ParallelScope.All` does +the same thing on a test method, because the suite itself always runs on its fixture's thread and only the `Children` +part has an effect. Whether the cases also overlap with other tests depends on the fixture. In a fixture that is not +itself parallel, the cases run in parallel with one another and nothing else runs at the same time, not even the +fixture's other tests. In a parallel fixture, the cases share the worker pool with everything else that is eligible. [!code-csharp[ParallelTestCases](~/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs#ParallelTestCases)] diff --git a/docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs b/docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs index af254b47a..5b702ecfa 100644 --- a/docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs +++ b/docs/snippets/Snippets.NUnit/ParallelTestsGuideExamples.cs @@ -125,7 +125,7 @@ public class SlugTests [TestCase("Hello World", "hello-world")] [TestCase(" NUnit ", "nunit")] [TestCase("A/B", "a-b")] - [Parallelizable(ParallelScope.All)] + [Parallelizable(ParallelScope.Children)] public void ToSlug_NormalizesInput(string input, string expected) { var slug = input.Trim().ToLowerInvariant().Replace(' ', '-').Replace('/', '-'); @@ -145,7 +145,7 @@ public class RateLimitedServiceTests [TestCase(2)] [TestCase(3)] [TestCase(4)] - [Parallelizable(ParallelScope.All)] + [Parallelizable(ParallelScope.Children)] public async Task Service_HandlesRequest(int request) { await Throttle.WaitAsync(); @@ -232,7 +232,7 @@ public void AllOnPlainTestMethod_IsNotRunnable() } [Test] - public void AllOnParameterizedTestMethod_IsRunnable() + public void ChildrenOnParameterizedTestMethod_IsRunnable() { var method = new MethodWrapper(typeof(SlugTests), nameof(SlugTests.ToSlug_NormalizesInput));