Skip to content

Parallelism guide/FAQ #272

Closed
Closed

Description

@jnm2

We've seen a few examples of folks asking why parallelism isn't working for them and it turns out that they're not realizing that they're sharing mutable state between threads. Sometimes it's something global and static, sometimes it's mutable fixture instance state because they expect a new fixture instance per test case. It would be good to provide examples of how to keep state separated, and what not to do (SetUp/TearDown), and why.

Here's an overly-complex example for a real-world project where I refactored from SetUp/TearDown toa parallel-safe solution: nunit/nunit#2105 (comment)

Activity

  1. AGBrown commented on Jun 18, 2018

    @AGBrown

    I'd like to suggest some commentary about enabling parallel-by-default at the test assembly level using [assembly: Parallelizable(...)] (in particular whether it should be [assembly: Parallelizable(ParallelScope.All)] or [assembly: Parallelizable(ParallelScope.Children)] or not).

    I guess this is clear from other statements that say only to use Self, Fixtures or Children (and not All) but it takes a bit of figuring out to reach that conclusion.

  2. CharliePoole commented on Jun 18, 2018

    @CharliePoole
    Member

    This will be a tricky thing to write because there are wide ranges of opinion as to the best way to use ParallelizableAttribute at assembly-level, or even whether to use it at all. I feel strongly that writeups about how NUnit works should be separate from subjective advice about how to best use it.

  3. AGBrown commented on Jun 28, 2018

    @AGBrown

    This will be a tricky thing to write because there are wide ranges of opinion as to the best way to use ParallelizableAttribute at assembly-level, or even whether to use it at all.

    I defer to the experts on this. Would it be accurate (and sufficiently un-opinionated) to describe the behaviour without making any recommendation with something like the following?

    "Applying [assembly: Parallelizable(ParallelScope.Children)] will result in all fixtures, child tests, and test cases/[Values] running in parallel, unless they in turn have Parallelizable attributes applied to override this behaviour."?

  4. jnm2 commented on Jun 28, 2018

    @jnm2
    ContributorAuthor

    Some folks will be interested in doing that, so that would be a valuable description. I wouldn't necessarily call out [Values] in particular.

  5. SeanKilleen commented on Oct 23, 2026

    @SeanKilleen
    Member

    A pass through nunit/nunit GitHub Discussions turned up six threads whose answers would fit this FAQ. They go beyond the shared-mutable-state point in the original issue, so listing them here as candidate entries:

    1. "Can I run namespaces in parallel but tests within a namespace serially?" No namespace-level attribute exists. The supported pattern is a [Parallelizable] [SetUpFixture] per namespace with the contained fixtures marked [NonParallelizable]. That pattern is described in Framework Parallel Test Execution case 4, but parallelizable.md does not link it, and the user in nunit/nunit#4679 only got it working after a 13-comment thread (working repro from @Bartleby2718 in that thread). Also nunit/nunit#4622.
    2. "Can I stop fixture A and B running together but let both run alongside C and D?" Not supported within one assembly. Answer in nunit/nunit#4657: split into assemblies and let dotnet test run assemblies in parallel via MaxCpuCount in runsettings.
    3. "Can I set LevelOfParallelism on one test?" No, assembly-only. Throttle inside the test with a static SemaphoreSlim. nunit/nunit#5039.
    4. "Why did adding Parallelizable make my run slower?" Thread start/stop cost dominates for tiny tests; and assembly-level parallelism is the test host's job (MaxCpuCount), not NUnit's. nunit/nunit#4466, which also surfaced the adapter AssemblySelectLimit default of 2000 as a hidden cliff.
    5. "My Selenium tests interfere with each other in parallel." The static-driver case: use FixtureLifeCycle(InstancePerTestCase) or a per-test driver. nunit/nunit#4634. This one is the original issue's point with a concrete example.
    6. "Parallelizable doesn't seem to do anything." @CharliePoole's line from Parallelization and setup and teardown pages need to mention mutual exclusiveness #242 is a good opener for the page: "Parallelizable doesn't actually tell NUnit to do anything. Rather, it just makes a promise to NUnit that your tests are capable of running in parallel."

    Smallest useful step regardless of the FAQ: add a "See also" link from parallelizable.md to the Framework Parallel Test Execution note, and a one-line note on levelofparallelism.md that per-test throttling has to be done in code.

  6. OsirisTerje commented on Oct 23, 2026

    @OsirisTerje
    Member

    Should we add a seperate FAQ section ? And place the different subjects below that ?
    Or should we integrate this into the relevant sections ?
    Or, should we do both ?

  7. SeanKilleen commented on Oct 23, 2026

    @SeanKilleen
    Member

    I'd lean toward both, with "FAQ section" meaning a topic page rather than a new top-level bucket.

    Your home-page overhaul in #1246 already set the pattern: "Flaky and Slow Tests" and "Tests That Depend on Other Tests" are question-shaped topic pages under Writing Tests, each answer is a few sentences plus a snippet, and they link out to the attribute reference pages. A "Running Tests in Parallel" page in the same style, with the questions above as its sections, is the common location. It gets a home-page card and sits next to Flaky and Slow Tests in the TOC.

    Then each of the Parallelizable, NonParallelizable and LevelOfParallelism reference pages gets a See also link to that page and to the Framework Parallel Test Execution technical note (none of them linked to it), plus a one-line pointer where the question naturally comes up, e.g. in the LevelOfParallelism notes that per-test throttling has to be done in code.

    The technical note stays as the "how it works" write-up, which keeps the separation @CharliePoole asked for in 2018: reference page = API, topic page = supported patterns described factually, technical note = scheduler internals.

    I'd rather not add a standalone FAQ section: nothing in the docs uses one today, and a section with one subject in it tends to become an orphan. If more question-shaped content turns up later, more topic pages scale better than a FAQ bucket.

    I've put that shape up as #1300. Two things surfaced while verifying the answers against NUnit 5 that are worth knowing even if the PR changes:

    • Case 4 in Framework Parallel Test Execution (parallel SetUpFixture with non-parallel fixtures runs as a group in parallel with other groups) doesn't hold. Every non-parallel fixture queues on the single non-parallel worker whatever namespace it is in, so the fixtures run one at a time. That's what you saw with the console runner in Tests in parallel fixtures not running in parallel per namespace nunit#4679. The PR corrects the case text.
    • ParallelScope.All on a non-parameterized test method is invalid (it's Self | Children), but valid on a [TestCase]/[TestCaseSource] method, where it runs the cases in parallel. The reference table said All was valid on methods.
  8. added a commit that references this issue on Oct 24, 2026
    d237333
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions